array-formulas on S Anand

Dynamically eliminate duplicates

Thu, 04 Sep 2008 12:00:00 +0000

This is a series on what Google Spreadsheets can do that Excel can’t

To get a list of unique values from a list, use the UNIQUE function on Google Spreadsheets.

For example, if you have a list of browsers in column A, type =UNIQUE(A1:A17) at cell B1 to get a unique list of browsers. This is a dynamic list. If you change the list of browsers, the unique list gets updated automatically.

You can use UNIQUE to create a dynamic pivot table. Quite often, you end up creating a pivot table simply to summarise by one column. The main purpose the pivot table serves is in getting a list of unique values on that column. Plus, it’s a bit heavy on the UI. And every time the data changes, you need to refresh the pivot. But with the UNIQUE function, you can get a dynamic list of unique values, and you can use the COUNTIF and SUMIF function next to each value. Here is an example showing the frequency table of the browsers shown earlier. Column C does a COUNTIF of the unique values on the original list.

You can also use UNIQUE as the input to another formula:

=COUNT(UNIQUE(LIST)) counts the number of unique values

=COUNT(LIST)-COUNT(UNIQUE(LIST)) gives the number of duplicates

=INDEX(UNIQUE(LIST),3) gives you the third unique value

=LARGE(UNIQUE(LIST),3) gives you the third largest unique value

… and so on.

Can I do that in Excel?

You can, but not easily. There are two approaches, but each has its limitations.

A. Use Advanced Filters: easy but static

Create an advanced filter on column A (Alt-D-F-A)
Select Copy to another location
Click in the Copy to box, and then click the cell B1
Select Unique records only
Click OK

But the list of unique values that you get here is static. If you changed one of the values, the list of unique values does not change.

B. Use a complex formulae that are dynamic

First, blank out the duplicates by typing this formula:

=IF(COUNTIF(A$1:A1,A1)=1,A1,"")

adjacent to the first cell (into B1), and dragging it all the way down (to B17).

Now, create a named range (Alt-I-N-D) for these cells (B1:B17) called WithBlanks and another named range called NoBlanks for the cells one column to the right (C1:C17).

On the first cell of NoBlanks (C1), type this formula:

=IF(ROW()-ROW(NoBlanks)+1>ROWS(WithBlanks)-COUNTBLANK(WithBlanks),"",
    INDIRECT(ADDRESS(SMALL((IF(WithBlanks<>"",ROW(WithBlanks),ROW()+
    ROWS(WithBlanks))),ROW()-ROW(NoBlanks)+1),COLUMN(WithBlanks),4)))

Press Ctrl-Shift-Enter rather than Enter, because it’s an array formula. Now drag this all the way down (to C17).

The list in column C is dynamic. If you change a cell in column A, column C is updated. But the formula can only handle one column. Google Spreadsheets’ UNIQUE function works with any number of columns. If you had data in the range A1:D100 and wanted the unique rows, UNIQUE(A1:D100) gets that for you.

Note: I’m staying away from user defined functions. You could, of course, create a UNIQUE function in Excel using Visual Basic. In fact, you should!

Comments

Ken J 19 Sep 2008 1:10 pm: Great entry. I love your site. It is perfect for advanced users.

Let’s hope Google’s pressure will force some innovation at excel.
hk 21 May 2009 3:30 am: What about Data/Consolidate from menu?

User defined array functions in Excel

Fri, 23 Feb 2007 12:00:00 +0000

Many languages have functions to process lists (array). These functions usually return a list, so you can pass that to another list function. This chaining of functions is really powerful.

UNIX provides this sort of chaining capability. If I had a cities (with some repetitions) and I wanted to find out how many started with the letter ‘A’, I’d just type:

cat cities | sort | uniq | grep "^A" | wc

cat: types the cities.
sort: sorts the cities alphabetically.
uniq: finds unique cities (works only if sorted - that's why we had to sort the list).
grep: filters the cities. Only allows cities beginning with A.
wc: word count

To do this on Excel, the only way is to

get the unique values. Data - Filter - Advanced Filter, and select “Unique records only”, “Copy the list to another location”, and select a location
get the first letter. =LEFT(cell,1) returns the first letter of the cell.
count the number of “A"s. =COUNT(range, “A”) counts the number of “A"s.

But ideally, I’d like a 1-line formula like

=LENGTH(UNIQUE(GREP("^A", range)))

Excel doesn’t provide these functions by default, but you can add them as user defined functions. Doing this lets you condense several cells into one. Instead of having to copy all your data into a set of unique values, and then adding a column for the first cell, the entire operation can be condensed into one formula.

I consider the following functions the a basic set for list processing.

LENGTH(list) counts the number of elements in a list
INDEX(list, n) returns the nth element of the list
GREP(string, list) returns elements of the list that have the string
UNIQUE(list) filters unique values
UNION(list, list) returns elements in at least one of the lists
INTERSECTION(list, list) returns elements in both lists
DIFFERENCE(list, list) returns the elements in the first list but not the second
REVERSE(list) reverses the order of the list
STRJOIN(separator, list) joins the elements of the list into a string, separated by a separator
STRSPLIT(separator, string) splits the string into a list, using a separator
MVLOOKUP(value, lookup, result) looks up value in “lookup”, and returns the corresponding MULTIPLE values from “result”

I created these UDFs. You can download the functions and play with them. Below are some tasks that you can do with them, that are difficult otherwise.

Get the file name from a path.

=INDEX(REVERSE(STRSPLIT(”", filename)), 1)

- Count the number of unique elements in a range.
  ```excel
=LENGTH(UNIQUE(range))

How many common elements are there in range 1 and range 2?

=LENGTH(INTERSECTION(range1, range2))

- How many words are there in a string?
  ```excel
=LENGTH(STRSPLIT(" ", string))

Get the smallest unique numbers in a range

=SMALL(UNIQUE(range), 5)

- Count the number of mismatches between two lists.
  ```excel
=COUNT(Range1)+COUNT(Range2) - COUNT(INTERSECTION(Range1,Range2))

Get a list of mismatches between two lists.

=STRJOIN(”,",UNION(DIFFERENCE(Range1,Range2), DIFFERENCE(Range2,Range1)))

- Count duplicate entries in a range.
  ```excel
=LENGTH(Range)-LENGTH(UNIQUE(Range))

VLOOKUP multiple values

=MVLOOKUP(A1,Lookup_Range,Return_Range)

- Count the unique matches in a VLOOKUP
  ```excel
=COUNT(UNIQUE(MVLOOKUP(A1,Lookup_Range,Return_Range)

This is a small sample. The power of list processing is phenomenal, especially when combined with array formulas. Download these macros and play with them!

Comments

Krishna 24 Feb 2007 4:10 am: Anand, thanks for an extremely useful stuff.In connection to our discussion on enhancing my excel skills, i think it would help me largely. Krishna
Sundar 26 Feb 2007 4:06 am: Very neat job Anand. Congrats for a super job. Waht is your e-mail address? Thanx in advance…………S.
Anand Srini 26 Feb 2007 9:40 pm: Hi I reproduce this extract from your article “Instead of having to copy all your data into a set of unique values, and then adding a column for the first cell, the entire operation can be condensed into one formula” can you explain why a column has to be added for the first cell ?
ashwin 27 Feb 2007 4:41 pm: Dear Anand, As usual; another useful post from you. Thanks, Ashwin
Kabir 5 Mar 2007 9:01 pm: you truly are a genius
Shiva 6 Mar 2007 2:03 pm: Your site is turning to a Dexters Laboratory
Ramly 19 Mar 2007 4:35 pm: Thank you very much this will help a lot
Mansi 11 Apr 2007 4:22 am: Hi, was just randomly searching for how to generate unique values on excel.. and came across your website. Must say, truly impressive work and this has helped me a lot! Thank you and keep up the good work! :)
James 23 Feb 2007 12:00 pm: Cheers, a lot of great stuff here, makes my Excel work much easier. My hat off to you!
Pete Watkins 23 Feb 2007 12:00 pm: Thanks ever so much for your work in developing these functions, as these have allowed me to avoid a huge amount of drudgery in processing data!

User-defined functions to get cell formatting

Sat, 02 Sep 2006 12:00:00 +0000

Sometimes you want to check the colour of a cell, or whether a cell is bold. This can be easily done with user-defined functions (UDFs). (To create a UDF, press Alt-F11, Alt-I-M, and type the code below.)

You can use ISBOLD(cell) to check if a cell is bold, and BGCOLOR(cell) to get the background colour of the cell. This lets you selectively process bold or shaded cells. The two examples below show how you can add only the cells in bold, or only the shaded cells.

Rather than use an additional column for ISBOLD or BGCOLOR, you can use an array formula, like below. (Remember to press Ctrl-Shift-Enter instead of Enter after typing this formula)

But first, you need to change the UDF to return an array rather than a single value. So IsBold will have to be modified as shown.

User-defined functions that process arrays can be very powerful. It can bring the full power of functional programming into Excel. I’ll describe some next week.

PS: In case you’re wondering, Application.Volatile tells Excel to recalculate the function every time the worksheet is recalculated. When a cell is made bold, or shaded, the value of the cell doesn’t change. So Excel doesn’t recalculate any formulas. You’ll have to manually press F9 every time to recalculate these cells. And Application.Volatile ensures that when you press F9, these functions are recalculated.

Comments

Ramly 19 Mar 2007 3:07 pm: good… May i ask your help to give me a solution for my problem… I always use vlookup to find some data in a specific range, my lookup value are employee code number then look for there name, age, address and so on. but today, i want look up a value with multiple returns. i want to have the some of a worker which he have in a specific date. Please give help regarding this matter.
Prasath 13 Nov 2009 10:00 am: Hi Need a help The ISBOLD function which is defined on top does not work if I change the cell which is Bold to Unbold. The formula should work automatically if a cell is changed BOLD or UNBOLD

Excel - Avoid manual labour 5

Wed, 21 Dec 2005 12:00:00 +0000

A few tips, without getting into the details. Array formulas let you perform any calculation on an entire list, and get the result in a single formula. Database functions like DSUM can manipulate tables of data very flexibly. Using these and Pivot tables, you can do anything with any tabular data.

Comments

Ananth 22 Dec 2005 6:32 am: I needed your feedback on www.doondo.org. Give it a spin let me know. I wanted to avoid positng here but never got any replies for the mails sent at root_node at yahoo.com
Ananth 22 Dec 2005 6:33 am: www.doondo.org is a hobby project I’ve been working on. The beta is just out.
Ananth 22 Dec 2005 6:35 am: Looking forward for your views and feedback
ritzkini 22 Dec 2005 10:01 am: great concept ananth..shld be a hit considering the great response to a similar funda used in IIM-I’s Klueless
S Anand 22 Dec 2005 12:31 pm: I liked the concept and interface, Ananth. Will mail you in detail. Quick question… how do I find out what the right keywords are?
ritzkini 23 Dec 2005 2:19 am: hehehe…hey anand,saw the visitor stats thing for the first time today ! awesomely cool ,kudos! u did it yourself ??
Dhar 23 Dec 2005 3:26 am: Ananth, does your game support advanced Google operators?
Ananth 23 Dec 2005 4:11 am: Thanks for feedback folks. doondo needs additional features, like game summary with right keywords. Right now only basic operators are supported. Even such a simple concept requires complex coding skills! I would love if you can post the suggestion at doondo@googlegroups.com. Great if you can join the group. Anand wouldn’t like spamming his comments section with doondo.
S Anand 24 Dec 2005 1:42 am: Yeah ritzkini, spent a week coding the visitor stats. What else do you think would be useful to see?
ritzkini 26 Dec 2005 7:22 am: :) i dont think it could have been any more comprehensive than it already is ! :)
harish 11 Jan 2006 3:50 am: great work mate…