{"id":1858,"date":"2013-10-31T20:04:05","date_gmt":"2013-10-31T20:04:05","guid":{"rendered":"http:\/\/www.sitekickr.com\/blog\/?p=1858"},"modified":"2013-11-01T01:46:03","modified_gmt":"2013-11-01T01:46:03","slug":"mysql-group_concat","status":"publish","type":"post","link":"https:\/\/www.sitekickr.com\/blog\/mysql-group_concat\/","title":{"rendered":"MySQL Group Concatenation &#8211; The hidden function"},"content":{"rendered":"<p>I admit, my work life is pretty easy. &nbsp;I work from home, I have awesome clients and a really great thermos that keeps my coffee warm for about 2 days. But that doesn&#39;t stop me from trying to make things easier, searching for the path of least resistance that will shave minutes off of any development task. That&#39;s why I&#39;m a big fan of the MySQL GROUP_CONCAT function.<br \/>\n\t&nbsp;<\/p>\n<h2>My Philosophy on GROUP_CONCAT<\/h2>\n<p>Before I get into detail, I thought I&#39;d share my philosophy on the MySQL Group_Concat function:<\/p>\n<ol>\n<li>People like lists, and Group_Concat is great at turning data into lists.<\/li>\n<li>It makes server-side processing easier, no loops required in simple cases.<\/li>\n<li>It can turn multiple queries into a single query (eliminating nested loops).<\/li>\n<li>It&#39;s more efficient &#8211; lists have smaller footprints than matrices.<\/li>\n<li>I can produce dangerously long result rows if you&#39;re not careful.<\/li>\n<li>It&#39;s not part of standard SQL, so it isn&#39;t well known.<\/li>\n<\/ol>\n<p>Before I get into some kinda cool examples, I just want to be sure reader&#39;s understand what this aggregate function does.<br \/>\n\t&nbsp;<\/p>\n<h2>A Brief&nbsp;GROUP_CONCAT Intro<\/h2>\n<p>You probably know about other aggregate functions:<\/p>\n<ul>\n<li>SUM gets the sum of all values in a group<\/li>\n<li>COUNT returns the number of items in the group<\/li>\n<li>AVG grabs the average of all values in the group<\/li>\n<\/ul>\n<p>We use these functions in conjunction with the GROUP BY statement. I&#39;ll stop there, you&#39;re not looking for a basic lesson in SQL here. So let me introduce the GROUP_CONCAT function.<\/p>\n<p>GROUP_CONCAT is an aggregate function like the others. And, like the other aggregate functions, it returns only one value. The big difference though is that the value it returns is actually a string, a string that is a concatenation of all values in the group.<\/p>\n<p>Let&#39;s say you have a table of <em><strong>users<\/strong><\/em>, and each user can have one or more orders, stored in an <em><strong>orders <\/strong><\/em>table.<\/p>\n<p>You are able to aggregate a list of all of a user&#39;s order numbers into one field, using GROUP_CONCAT.<\/p>\n<pre class=\"prettyprint\"><code>SELECT users.username, GROUP_CONCAT(orders.order_id) as ordernums\r\nFROM users\r\nINNER JOIN orders\r\n        ON orders.user_id = users.user_id\r\nGROUP BY users.user_id<\/code><\/pre>\n<p>This will return a result set similar to:<\/p>\n<p><strong>Username &nbsp; &nbsp; &nbsp; &nbsp;Ordernums<\/strong><br \/>\n\ttestuser1 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;5,16,32,103<br \/>\n\ttestuser2 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;78,101<br \/>\n\ttestuser3 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;104,106<\/p>\n<p>&nbsp;<\/p>\n<h2>Real World GROUP_CONCAT Examples<\/h2>\n<p>Now that you have an idea what GROUP_CONCAT does, for the purposes of solidifying that knowledge, let&#39;s have a look at some interesting ways to use it. These are real-life examples, ways I&#39;ve actually employed GROUP_CONCAT in my development.<br \/>\n\t&nbsp;<\/p>\n<h3>Direct to Semantic HTML<\/h3>\n<p>The HTML <em>definition list<\/em> (now called a <em>description list <\/em>in HTML 5) isn&#39;t used quite as often as it should be. I imagine it&#39;s a mix of it being more difficult to style than other tags and it&#39;s lack of understanding. It should be used more often to define relationships between a term and its definition or description. The MySQL GROUP_CONCAT function makes really easy work of directly outputting valid HTML in the form of a definition list.<\/p>\n<p>Let&#39;s say we have a table like this:<\/p>\n<p><strong>Name &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Job Title<br \/>\n\t<\/strong> Dan Aykroyd &nbsp; &nbsp; &nbsp; &nbsp;Vice-President<br \/>\n\tChevy Chase &nbsp; &nbsp; &nbsp; &nbsp;Account Executive<br \/>\n\tBilly Crystal &nbsp; &nbsp; &nbsp; &nbsp; Systems Analyst<\/p>\n<p>We&#39;re looking for a nice easy way to display a list of job titles, along with the person that holds that job title. We could use the traditional-style query that returns three result rows, or we could go with a query that aggregates all results into a single row, and formats it as HTML!<\/p>\n<pre class=\"prettyprint\"><code>SELECT \r\nGROUP_CONCAT(&#39;&lt;dt&gt;&#39;,name, &#39;&lt;\/dt&gt;&lt;dd&gt;&#39;, job_title, &#39;&lt;\/dd&gt;&#39; SEPARATOR &#39;\\n&#39;) as jobs\r\nFROM people\r\n<\/code><\/pre>\n<p>This will return a single row, with a single field that contains a nice little piece of HTML. HTML that we can directly output without any processing:<\/p>\n<pre class=\"prettyprint\"><code>&lt;dt&gt;Dan Aykroyd&lt;\/dt&gt;&lt;dd&gt;Vice-President&lt;\/dd&gt;\r\n&lt;dt&gt;Chevy Chase&lt;\/dt&gt;&lt;dd&gt;Account Executive&lt;\/dd&gt;\r\n&lt;dt&gt;Billy Crystal&lt;\/dt&gt;&lt;dd&gt;Systems Analyst&lt;\/dd&gt;<\/code><\/pre>\n<p>Of course, this type of use should be considered when a script depending on those query results is expecting HTML output.<\/p>\n<p>&nbsp;<\/p>\n<h3>Avoiding Multiple Queries and Nested Loops<\/h3>\n<p><a href=\"http:\/\/www.sitekickr.com\/blog\/wp-content\/uploads\/2013\/10\/group-concat-no-nested-loops.png\"><img loading=\"lazy\" alt=\"group_concat prevents nested loops\" class=\"alignnone size-full wp-image-1899\" height=\"256\" src=\"http:\/\/www.sitekickr.com\/blog\/wp-content\/uploads\/2013\/10\/group-concat-no-nested-loops.png\" width=\"512\" srcset=\"https:\/\/www.sitekickr.com\/blog\/wp-content\/uploads\/2013\/10\/group-concat-no-nested-loops.png 512w, https:\/\/www.sitekickr.com\/blog\/wp-content\/uploads\/2013\/10\/group-concat-no-nested-loops-300x150.png 300w\" sizes=\"(max-width: 512px) 100vw, 512px\" \/><\/a><\/p>\n<p>Another example of something I did for a client of mine just a week ago, though I changed the non-essential details for this example.<\/p>\n<p>A car dealership has an inventory of cars online, grouped by the make of the car. For instance:<\/p>\n<p><strong>Ford<\/strong><br \/>\n\t2010 F150<br \/>\n\t2008 Mustang<br \/>\n\t2012 Escape<\/p>\n<p>\t<strong>Toyota<\/strong><br \/>\n\t2005 Corolla<br \/>\n\t2008 4Runner<\/p>\n<p>\tBob, the website manager, has the job of accepting requests from various people at the dealership to list cars on the website as they come into the lot. Anyone can submit a listing request car by filling out a simple form (choosing the Make, Model, Year &amp; Price).<\/p>\n<p>The difficulty that Bob is having though, is that there is poor communication within the dealership and employees are submitting cars that have already been submitted by someone else.<\/p>\n<p>Bob needs a real quick way to see which cars have already been listed for a particular make, before approving a request.<\/p>\n<p>Enter GROUP_CONCAT&#8230;<\/p>\n<pre class=\"prettyprint\">SELECT \r\n   c.id, c.year, c.name, ct.name as category_name\r\n  (SELECT GROUP_CONCAT(&#39; &#39;, year, &#39; &#39;, name)\r\n   FROM cars\r\n   WHERE car_type_id = ct.id AND <strong>approved = 1<\/strong> \r\n   GROUP BY car_type_id) as already_approved_list\r\nFROM cars c\r\nINNER JOIN car_types ct\r\n        ON ct.id = c.car_type_id\r\nWHERE <strong>c.approved = 0<\/strong>\r\n<\/pre>\n<p>This query will produce a result set like this:<\/p>\n<pre>ID   YEAR  NAME      CATEGORY_NAME   ALREADY_APPROVED_LIST\r\n101  2012  Escape    Ford             2010 F150, 2008 Mustang, 2012 Escape\r\n102  2011  4Runner   Ford             2005 Corolla, 2008 4Runner\r\n<\/pre>\n<p>By looking at this list, Bob can instantly see that the 2012 Escape is a duplicate listing and should not be approved.<\/p>\n<blockquote><p>There is only one query required to obtain this information, and hence, only one loop required to display it.<\/p><\/blockquote>\n<p>&nbsp;<\/p>\n<h3>Concatenation with NULL is like multiplying by zero<\/h3>\n<p>In grade school, they taught you that any number multiplied by zero produces a result of zero. But, for some reason, your teacher forgot to mention that this also applies to MySQL string concatenation.<\/p>\n<blockquote><p>In MySQL (and other databases), any string concatenated with a <kbd>NULL <\/kbd>value produces a <kbd>NULL <\/kbd>result.<\/p><\/blockquote>\n<p>Let&#39;s illustrate with an example, building on the car inventory above:<\/p>\n<p><strong>Toyota<\/strong><br \/>\n\t2005 Corolla<br \/>\n\tNULL 4Runner<\/p>\n<pre>ID   YEAR  NAME      CATEGORY_NAME   ALREADY_APPROVED_LIST\r\n102  2011  4Runner   Ford             2005 Corolla\r\n<\/pre>\n<p>Note that the 4Runner isn&#39;t listed, even though its&#39; name is supplied. Because the year is <kbd>NULL<\/kbd>, the resulting concatenation of year and name equals <kbd>NULL<\/kbd>.<\/p>\n<p>That&#39;s outrageous, right! How can I possibly use the <strong><kbd>GROUP_CONCAT <\/kbd><\/strong>function if it doesn&#39;t work as expected when <kbd>NULL <\/kbd>values are introduced. <kbd>NULL <\/kbd>values are extremely common in my data.<\/p>\n<div class=\"post-aside\">\n<h4>Coalesce, is that the right name for it?<\/h4>\n<p>We learned in science class that the word coalesce means &quot;to come together or unite into one mass&quot;. As in when drops of water coalesce into a puddle. But that&#39;s not really what&#39;s going on with the MySQL <strong>Coalesce <\/strong>function. The Coalesce function returns the first non-<kbd>NULL <\/kbd>argument passed to it.<\/p>\n<p><a href=\"http:\/\/www.sitekickr.com\/blog\/wp-content\/uploads\/2013\/10\/coalesce.png\"><img loading=\"lazy\" alt=\"coalesce\" class=\"alignnone size-full wp-image-1907\" height=\"128\" src=\"http:\/\/www.sitekickr.com\/blog\/wp-content\/uploads\/2013\/10\/coalesce.png\" width=\"128\" \/><\/a><\/p>\n<p>In my opinion, coalescence is more closely related to string concatenation. I think it should have been Oracle&#39;s first priority, after buying MySQL, to do a deep investigation of who decided to name this function and why they never took a science class \ud83d\ude09<\/p>\n<\/div>\n<p>The MySQL development crew knows how important <kbd>NULL <\/kbd>values are, and how often they occur &#8211; so much so that they&#39;ve provided a function specifically for dealing with <kbd>NULL <\/kbd>values. This function is named <strong>Coalesce<\/strong>.<\/p>\n<p><strong>Coalesce <\/strong>will return the first non-NULL argument which is passed into it. If you haven&#39;t heard about this function before, you can see how useful it can be in conjunction with concatenation functions.<\/p>\n<p>It&#39;s use is simple. If we were to leverage Coalesce in our example above, it would look like this:<\/p>\n<pre class=\"prettyprint\" style=\"width: 320px;\">SELECT \r\n   c.id, c.year, c.name, \r\n   ct.name as category_name\r\n  (SELECT \r\n   GROUP_CONCAT(&#39; &#39;, <strong>COALESCE<\/strong>(year, &#39;&#39;), \r\n                &#39; &#39;, name)\r\n   FROM cars\r\n   WHERE car_type_id = ct.id AND \r\n         approved<strong> = 1<\/strong> \r\n   GROUP BY car_type_id) \r\n   as already_approved_list\r\nFROM cars c\r\nINNER JOIN car_types ct\r\n        ON ct.id = c.car_type_id\r\nWHERE <strong>c.<\/strong>approved <strong>= 0<\/strong><\/pre>\n<p>If the year field happens to contain a <kbd>NULL <\/kbd>value, the Coalesce function will compare it against the other values passed in and return the empty string provided to it.<\/p>\n<p>&nbsp;<\/p>\n<p>So that&#39;s my take on <kbd>GROUP_CONCAT<\/kbd>. Think about this function whenever you have nested query loops and you have that sneaking suspicion that something could be done a little bit better.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The MySQL GROUP_CONCAT function isn&#8217;t a well-known feature, but it is one of the most useful functions that exists in MySQL. I call it the &#8220;loop-killer&#8221;. Here are a few ways to put it to use on your site.<\/p>\n","protected":false},"author":1,"featured_media":1885,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"amp_status":""},"categories":[13],"tags":[88,93],"_links":{"self":[{"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/posts\/1858"}],"collection":[{"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/comments?post=1858"}],"version-history":[{"count":34,"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/posts\/1858\/revisions"}],"predecessor-version":[{"id":1918,"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/posts\/1858\/revisions\/1918"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/media\/1885"}],"wp:attachment":[{"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/media?parent=1858"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/categories?post=1858"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sitekickr.com\/blog\/wp-json\/wp\/v2\/tags?post=1858"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}