157 Matching Annotations
  1. Aug 2024
    1. ELECT DISTINCT ON (customer) id, customer, total FROM purchases ORDER BY customer, total DESC, id;
    2. The perfect index for the above query would be a multi-column index spanning all three columns in matching sequence and with matching sort order:
    1. WITH recursive temp (n, fact) AS ( SELECT 0, 1 -- Initial Subquery UNION ALL SELECT n+1, (n+1)*fact FROM temp WHERE n < 9 -- Recursive Subquery ) SELECT * FROM temp;
  2. Apr 2024
    1. select count(*) filter (where year <= 1945) as pre1945, count(*) filter (where year between 1946 and 1964) as period2, count(*) filter (where year between 1965 and 1974) as period3, ... from ... where ...;
  3. Jan 2024
  4. Oct 2023
  5. Jul 2023
  6. Jun 2023
    1. Writing json_populate_record in the FROM clause is good practice, since all of the extracted columns are available for use without duplicate function calls.
  7. May 2023
  8. Apr 2023
    1. SELECT CNPJ, razaoSocialFROM EmpresaWHERE atividade = '4120-4/00

      Nesse comando, a tabela "Empresa" é selecionada e as colunas "CNPJ" e "razaoSocial" são especificadas como resultado da consulta. A cláusula WHERE filtra apenas as linhas da tabela "Empresa" que têm a coluna "atividade" igual a '4120-4/00', ou seja, as empresas cuja atividade econômica principal é a construção de edifícios.

    2. SELECT E.CNPJ, E.razaoSocialFROM Empresa EWHERE E.CNPJ NOT IN (SELECT T.CNPJFROM Tem TJOIN Caracteristica C ON (T.cod = C.cod)WHERE C.sigla = 'ESG')

      O código SQL acima faz uma consulta em duas tabelas, Empresa e Tem, para retornar apenas as empresas que não têm a característica ESG (governança ambiental, social e corporativa).

      A cláusula WHERE filtra as empresas que não estão presentes na subconsulta. A subconsulta retorna os CNPJs das empresas que têm a característica ESG, utilizando a tabela Tem, que relaciona as características com as empresas por meio dos campos CNPJ e cod. A cláusula JOIN é usada para juntar as tabelas Tem e Caracteristica.

      Assim, a consulta principal retorna apenas os CNPJs e razões sociais das empresas que não estão na subconsulta, ou seja, que não têm a característica ESG.

  9. Mar 2023
  10. Feb 2023
    1. Most applications do not need to process massive amounts of data. This has led to a resurgence in data management systems with traditional architectures; SQLite, Postgres, MySQL are all growing strongly, while “NoSQL” and even “NewSQL” systems are stagnating.

      SQL still shines over NoSQL

  11. Jan 2023
  12. Dec 2022
    1. In the Containers pane, expand all nodes of the package container hierarchy, and then clear all check boxes, including the Extract Sample Currency Data check box. Now select the check box for Extract Sample Currency Data to get only the events for this node.

      When I completed this step, the nodes of the package container hierarchy did not have checkboxes. Also, the Extract Sample Currency Data checkbox did not respond to my clicks.

      There was a message to clear the parent, but since the parent had no checkbox, I was unable to do so.

      Drilling down resulted in checkboxes that "appear dimmed" as in the note below.

      This appears to be a bug or at least a lapse in the documentation.

      EDIT: We found a work-around. You can highlight the checkbox and tick it with the spacebar - but you can't select it with the mouse.

      On the Lesson 3 root node, under Select the logs to use for the container: I was able to select the checkbox.

      When I navigate deeper though, this option is unchecked and muted or grayed out with a message at the bottom reading:

      "⚠️ To configure unique logging options for this container, enable logging for it in the tree view."

  13. Nov 2022
    1. Post.in_order_of(:type, %w[Draft Published Archived]).order(:created_at).pluck(:name) which generates SELECT posts.name FROM posts ORDER BY CASE posts.type WHEN 'Draft' THEN 1 WHEN 'Published' THEN 2 WHEN 'Archived' THEN 3 ELSE 4 END ASC, posts.created_at ASC
  14. Oct 2022
    1. Many popular SQL clients do not use SSL by default. If you aren’t deliberate about choosing encryption, the connection will be unencrypted.

      Table with SQL clients and their default SSL mode:

  15. Aug 2022
  16. Jul 2022
    1. this application uses SQLAlchemy, an ORM library for Python that allows the developer to create schemas and queries using mostly database-independent Python constructs.

      How if compared with SQLModel?

    2. In my opinion , SQLite is easy/small to host on Cloud platform but PostgreSQL might be difficult if compared. For larger data , SQLite is not suitable.

    1. How does PostgreSQL access a column? It will fetch the row and then dissect this tuple to calculate the position of the desired column inside the row. So if we want to access column #1000 it means that we have to figure out how long those first 999 columns before our chosen one really are. This can be quite complex. For integer we simply have to add 4, but in case of varchar, the operation turns into something really expensive.

      Column order matters in PostgreSQL. The time to read 1000th column can be pretty large, especially if there are varchar columns before that.

  17. Jun 2022
  18. May 2022
    1. Create the new empty table Write to both old and new table Copy data (in chunks) from old to new Validate consistency Switch reads to new table Stop writes to the old table Cleanup old table

      7 steps required while migrating to a new table

    1. COUNT(*)表示查询所有列的行数,要注意聚合的计算结果虽然是一个数字,但查询的结果仍然是一个二维表,只是这个二维表只有一行一列,并且列名是COUNT(*)

      聚合操作我的理解是“给数据降维

    1. IF NOT EXISTS (SELECT * FROM spatial_ref_sys) THEN ⋮ ANALYZE spatial_ref_sys;
  19. Apr 2022
    1. The lateral keyword allows us to access columns after the FROM statement, and reference these columns "earlier" in the query ("earlier" meaning "written higher in the query").
    1. The difference between a non- lateral and a lateral join lies in whether you can look to the left hand table's row.
    2. The comma ( , ) in the FROM clause is short notation for CROSS JOIN . LATERAL is assumed automatically for table functions.
    1. LEFT OUTER JOIN First, an inner join is performed. Then, for each row in T1 that does not satisfy the join condition with any row in T2, a joined row is added with null values in columns of T2. Thus, the joined table always has at least one row for each row in T1.
    2. A LATERAL item can appear at top level in the FROM list, or within a JOIN tree. In the latter case it can also refer to any items that are on the left-hand side of a JOIN that it is on the right-hand side of.

      Unlike with most joins (IIUC), order is important with lateral joins. Weird. Why?

      Maybe because it is equivalent to a cross join lateral (see example), and in an explicit cross join, you have a LHS and RHS?

    3. This allows them to reference columns provided by preceding FROM items.
    4. It is often particularly handy to LEFT JOIN to a LATERAL subquery, so that source rows will appear in the result even if the LATERAL subquery produces no rows for them.
    1. Why not just use a join and group by? SELECT AA.ID, COUNT(B.ID) as no_tx, min(B.DATE) as fday_tx, max(B.DATE) as lday_tx, AA.start_date, AA.end_date FROM (SELECT ID, min(DATE) as start_date, max(DATE) as end_date FROM MAIN_TABLE WHERE CODE = 'drugA' GROUP BY ID ) AA LEFT JOIN MAIN_TABLE b ON b.CODE = 'drugB' AND b.DATE > AA.start_date AND b.DATE < AA.end_date GROUP BY AA.ID, AA.start_date, AA.end_date;
    2. window functions: SELECT ID, SUM(CASE WHEN code = 'drugB' THEN 1 ELSE 0 END) as no_tx, MIN(CASE WHEN code = 'drugB' THEN DATE END) as fday_tx, MIN(CASE WHEN code = 'drugB' THEN DATE END) as lday_tx, start_date, end_date FROM (SELECT t.*, MIN(CASE WHEN code = 'drugA' THEN date END) as start_date, MAX(CASE WHEN code = 'drugB' THEN date END) as end_date FROM MAIN_TABLE t ) t WHERE code in ('drugA', 'drugB') AND date between start_date and end_date GROUP BY t.id;
    1. The query to the right of the Lateral would be evaluated for every row of the left table.
    1. SELECT lateral_subquery.* FROM posts JOIN LATERAL ( SELECT comments.* FROM comments WHERE (comments.post_id = posts.id) LIMIT 3 ) lateral_subquery ON true WHERE posts.id
    2. You want the front page to show a few hundred posts along with the top three comments on each post. You’re planning on being very popular, so the front page will need to be very fast. How do you fetch that data efficiently from postgresql using Activerecord?
    3. Making one Comment query per Post is too expensive; it’s N+1 queries (one to fetch the posts, N to fetch the comments). You could use includes to preload all the comments for all the posts, but that requires hydrating hundreds of thousands of records, even though you only need a few hundred for your front page. What you want is some kind of GROUP BY with a LIMIT on each group — but that doesn’t exist, either in Activerecord nor even in postgres. Postgres has a different solution for this problem: the LATERAL JOIN.
    1. Description of the problem: Select all users from a DB with their parents which have the association by parent_id column. Possible solution: Recursive Common Table Expression.
    1. join = Arel::Nodes::NamedFunction.new('json_b_array_elements', [Arel::Nodes::SqlLiteral.new("subscriptions")]) .as(Arel::Nodes::NamedFunction.new('sd', [Arel::Nodes::SqlLiteral.new("subscription_data")]).to_sql) p = e.project( Arel::Nodes::SqlLiteral.new( Arel::Nodes::Grouping.new( Arel::Nodes::InfixOperation.new('->>', sd[:subscription_data], Arel::Nodes::SqlLiteral.new("'id'"))).to_sql) << '::uuid' ).where( Arel::Nodes::InfixOperation.new('->>', sd[:subscription_data], Arel::Nodes::SqlLiteral.new("'type'").eq( Arel::Nodes::SqlLiteral.new("'Company'") ) ).and(e[:slug].eq(event_slug))) p.join_sources << Arel::Nodes::StringJoin.new( Arel::Nodes::SqlLiteral.new('CROSS JOIN LATERAL')) << join
    1. SELECT "users".* FROM "users" wHERE 'admin' = ANY("users"."roles")
    2. any_role = Arel::Nodes::NamedFunction.new("ANY", [User[:roles]])

      any

  20. Mar 2022
  21. Feb 2022
    1. lateral (select pledged / fx_rate as pledged_usd) pu, lateral (select pledged_usd / backers_count as avg_pledge_usd) apu, lateral (select goal / fx_rate as goal_usd) gu, lateral (select goal_usd - pledged_usd as usd_from_goal) ufg, lateral (select (deadline - launched_at)/86400.00 as duration) dr
  22. Nov 2021
    1. It's redundant. The transaction commit or rollback can happen in the C# code or the sproc but not both. 

      c# / SQL transactions

  23. Oct 2021
    1. So, what’s a better way to illustrate JOIN operations? JOIN diagrams!

      Apparently, SQL should be taught using JOIN diagrams not Venn diagrams?

  24. Jul 2021
    1. The most important part of this query is the with block. It's a powerful resource, but for this example, you can think of it as a "way to store a variable" that is the path of the contact you need to update, which will be dynamic depending on the record.
    2. It just builds the path as '{1, value}', but we need to convert to text[] because that’s the type expected on the jsonb_path function.
    1. In addition to SQLAlchemy core queries, you can also perform raw SQL queries

      Instead of SQLAlchemy core query:

      query = notes.insert()
      values = [
          {"text": "example2", "completed": False},
          {"text": "example3", "completed": True},
      ]
      await database.execute_many(query=query, values=values)
      

      One can write a raw SQL query:

      query = "INSERT INTO notes(text, completed) VALUES (:text, :completed)"
      values = [
          {"text": "example2", "completed": False},
          {"text": "example3", "completed": True},
      ]
      await database.execute_many(query=query, values=values)
      

      =================================

      The same goes with fetching in SQLAlchemy:

      query = notes.select()
      rows = await database.fetch_all(query=query)
      

      And doing the same with raw SQL:

      query = "SELECT * FROM notes WHERE completed = :completed"
      rows = await database.fetch_all(query=query, values={"completed": True})
      
  25. Jun 2021
    1. HAVING avg(score) < 0.75 * (SELECT avg(score) FROM performance_reviews)
    2. When you are dealing with an aggregate of aggregates, it needs to be accomplished in two steps. This can be done using a subquery as the FROM clause, essentially giving us a temporary table to then select from, allowing us to find the average of those averages.
    1. SELECT * FROM ( -- build virtual table of all hours between -- a date range SELECT start_ts, start_ts + interval '1 hour' AS end_ts FROM generate_series( '2017-03-01'::date, '2017-03-03'::timestamp - interval '1 hour', interval '1 hour' ) AS t(start_ts) ) AS cal LEFT JOIN ( -- build virtual table of uptimes SELECT * FROM ( VALUES ('2017-03-01 01:15:00-06'::timestamp, '2017-03-01 02:15:00-06'::timestamp), ('2017-03-01 08:00:00-06', '2017-03-01 20:00:00-06'), ('2017-03-02 19:00:00-06', null) ) AS t(start_ts, end_ts) ) AS uptime ON cal.end_ts > uptime.start_ts AND cal.start_ts <= coalesce(uptime.end_ts, current_timestamp)
  26. Apr 2021
  27. Mar 2021
    1. This is not a problem if your DBMS supports SQL recursion: lots of data can be generated with a single query. The WITH RECURSIVE clause comes to the rescue.

      WITH RECURSIVE can help you quickly generate a series of random data.

  28. Feb 2021
  29. Oct 2020
  30. Sep 2020
    1. The Realm is a new database module that is improving the way databases are used and also supports relationships between objects. If you are part of the SQL development world, then you must be familiar with the Realm.
  31. Jul 2020
    1. [:returning] (Postgres-only) An array of attributes that should be returned for all successfully inserted records. For databases that support INSERT ... RETURNING, this will default to returning the primary keys of the successfully inserted records. Pass returning: %w[ id name ] to return the id and name of every successfully inserted record or pass returning: false to omit the clause.
  32. Jun 2020
    1. NoSQL databases typically perform better and are easier to scale due to the nature of their data access and storage
  33. May 2020
    1. WHERE clause introduces a condition on individual rows; HAVING clause introduces a condition on aggregations, i.e. results of selection where a single result, such as count, average, min, max, or sum, has been produced from multiple rows. Your query calls for a second kind of condition (i.e. a condition on an aggregation) hence HAVING works correctly. As a rule of thumb, use WHERE before GROUP BY and HAVING after GROUP BY. It is a rather primitive rule, but it is useful in more than 90% of the cases.

      Difference between WHERE and HAVING in SQL:

      • WHERE is used for individual rows
      • HAVING is used for aggregations (result of a selection), such as: COUNT, AVERAGE, MIN, MAX or SUM
      • Use WHERE before GROUP BY and HAVING after GROUP BY (works in 90% of the cases)
    1. Which database technology to choose

      Which database to choose (advice from an Amazon employee):

      • SQL - ad hoc queries and/or support of ACID and transactions
      • NoSQL - otherwise. NoSQL is getting better with transactions and PostgreSQL is getting better with availability, scalability, durability
  34. Apr 2020
    1. 1) Redash and Falcon focus on people that want to do visualizations on top of SQL2) Superset, Tableau and PowerBI focus on people that want to do visualizations with a UI3) Metabase and SeekTable focus on people that want to do quick analysis (they are the closest to an Excel replacement)

      Comparison of data analysis tools:

      1) Redash & Falcon - SQL focus

      2) Superset, Tableau & PowerBI - UI workflow

      3) Metabase & SeekTable - Excel like experience

    1. Joins are not expensive. Who said it to you? As basically the whole concept of relational databases revolve around joins (from a practical point of view), these product are very good at joining. The normal way of thinking is starting with properly normalized structures and going into fancy denormalizations and similar stuff when the performance really needs it on the reading side. JSON(B) and hstore (and EAV) are good for data with unknown structure.
  35. Mar 2020
    1. Another nice SQL script paired with CRON jobs was the one that reminded people of carts that was left for more than 48 hours. Select from cart where state is not empty and last date is more than or equal to 48hrs.... Set this as a CRON that fires at 2AM everyday, period with less activity and traffic. People wake up to emails reminding them about their abandoned carts. Then sit watch magic happens. No AI/ML needed here. Just good 'ol SQL + Bash.

      Another example of using SQL + CRON job + Bash to remind customers of cart that was left (again no ML needed here)

    2. I will write a query like select from order table where last shop date is 3 or greater months. When we get this information, we will send a nice "we miss you, come back and here's X Naira voucher" email. The conversation rate for this one was always greater than 50%.

      Sometimes SQL is much more than enough (you don't need ML)

    1. ACID stands for Atomicity (an operation either succeeds completely or fails, it does not leave partial data), Consistency (once an application performs an operation the results of that operation are visible to it in every subsequent operation), Isolation (an incomplete operation by one user does not cause unexpected side effects for other users), and Durability (once an operation is complete it will be preserved even in the face of machine or system failure).

      ACID definition

  36. Jan 2020
    1. The SELECT part should not contain any columns not referenced in GROUP BY clause, unless it is wrapped with an aggregate function.
    1. Which to use? ANY is a later, more versatile addition, it can be combined with any binary operator returning a boolean value. IN burns down to a special case of ANY. In fact, its second form is rewritten internally: IN is rewritten with = ANY NOT IN is rewritten with <> ALL
    1. Single quotes return text strings.  Double quotes return (if you can really think of them as “returning” anything) identifiers, but with the case preserved.
  37. Dec 2019
  38. unix4lyfe.org unix4lyfe.org
    1. if you care at all about storing timestamps in MySQL, store them as integers and use the UNIX_TIMESTAMP() and FROM_UNIXTIME() functions.

      MySQL does not store offset

  39. Oct 2019
    1. Database engines in practice don’t actually run queries by joining, and then filtering, and then grouping, because they implement a bunch of optimizations reorder things to make the query run faster as long as reordering things won’t change the results of the query

      SQL queries are run by database engines in different order than we write them down

    2. SELECT isn’t the first thing, it’s like the 5th thing!

      Order of SQL queries:

      1. FROM/JOIN and all the ON conditions
      2. WHERE
      3. GROUP BY
      4. HAVING
      5. SELECT (including window functions)
      6. ORDER BY
      7. LIMIT SQL queries happen in this order* 1.
  40. Mar 2019
    1. DBA Por Acaso: RDS, MySQL e Tuning para Iniciantes

      Outro assunto que não é explicitamente coberto por nossos tópicos, mas que é base importante para o administrador de sistemas na nuvem - e aqui coberto em um nível introdutório, para não assustar ninguém. E procura por Cloud em https://wiki.lpi.org/wiki/DevOps_Tools_Engineer_Objectives_V1 para ver como esse assunto é importante!

  41. Dec 2018
    1. SELECT sj.name , sja.* FROM msdb.dbo.sysjobactivity AS sja INNER JOIN msdb.dbo.sysjobs AS sj ON sja.job_id = sj.job_id WHERE sja.start_execution_date IS NOT NULL AND sja.stop_execution_date IS NULL

      View current running jobs SQL