Dynamic arrays in Excel with Python and xlwings

The days of Ctrl-Shift-Enter and You can't change part of an array are finally over! Say hello to the new dynamic arrays that Microsoft has now (July 2019) rolled out to a broader user base: their Office 365 subscribers who are on the monthly channel.

To check if you are on the monthly channel, go to File > Account and see if it mentions Monthly Channel (Targeted) under About Excel. If you are not, then you’ll need to wait a bit longer (or switch to the monthly channel). Also note that dynamic arrays are not yet part of Office 2019, so you will absolutely need an Office 365 subscription.

To quickly double check if the respective update has already been installed, try if you have access to the =UNIQUE formula. If you do, you’re all set for Excel’s most revolutionary feature since the introduction of the ribbon and the xlsx format. If you don’t, go to File > Account > Office Updates and click on Update Now.

Dynamic arrays are obviously great news if you work with Excel in general, but they also make writing UDFs (User Defined Functions) in Python via xlwings a lot more powerful. This post assumes you know how xlwings UDFs work, so head over to the xlwings quickstart if you need a refresher.

Overview

1. What’s wrong with the legacy Ctrl-Shift-Enter array formulas?
2. The new dynamic array formulas and #Spill errors
3. xlwings UDFs and the new dynamic arrays
4. New built-in Excel formulas: UNIQUE etc.

Prefer to watch a video? Here it is:

What’s wrong with the legacy Ctrl-Shift-Enter array formulas?

Consider the following xlwings UDF:

import xlwings as xw
import numpy as np

@xw.func
def dynamic_array(r, c):
    return np.random.rand(int(r), int(c))

In older versions of Excel, if you write =dynamic_array(4, 3) into A1, then you would just get one value back instead of the full 4 x 3 array. To solve that, you’d have to use legacy array formulas: Select all cells for the result array (i.e. A1:C4 in the example), then, while the cells are selected, write the formula into the top left cell and finally hit Ctrl-Shift-Enter (see the xlwings docs for a step-by-step tutorial).

The biggest issue arises when you want to change the formula to =dynamic_array(5, 5): You have to first delete the entire array (you need to select the entire array to be able to do that), then select the new 5 x 5 cell array, type in the new formula and finally repeat the Ctrl-Shift-Enter madness.

Things get even more complicated when you don’t know the dimensions of the array up front: You might, for example, query an API or database to download time series data of unknown length and return this via your UDF.

xlwings offers a “hacky” solution for this issue: When you are on an older versions of Excel without dynamic arrays, you can use the @xw.ret(expand='table') decorator which will automatically resize the legacy array for you:

@xw.func
@xw.ret(expand='table')
def dynamic_array(r, c):
    return np.random.rand(int(r), int(c))

Still, this is hacky, slow and error prone so being able to let Excel take care of this properly is a game changer.

The new dynamic array formulas and #Spill errors

Whenever you use an Excel formula or xlwings UDF that returns an array, Excel will now automatically write out the whole array and put a blue line around it. No more difference to how you would type in a single cell formula and no more Ctrl-Shift-Enter! The curly braces around your formula that appear with legacy arrays are also gone.

If the array would overwrite existing cell values, it returns the new #Spill error instead, allowing you to make room for the array. Once you move or delete the offending cell, Excel returns the array automatically and gets rid of the #Spill error.

If you ever want to prevent a formula from behaving like a new dynamic array, you simply have to start the formula with an @, e.g.: =@dynamic_array(3,4) and you’re back to the old behavior.

xlwings UDFs and the new dynamic arrays

The best part about xlwings and dynamic arrays is that they work out of the box without any changes. However, if you were using the xw.ret(expand='table') decorator previously, make sure to remove it again as soon as you have upgraded to an Excel version that supports dynamic arrays!

For example, building a UDF for Quandl time series is now as easy as:

import xlwings as xw
import quandl

# quandl.ApiConfig.api_key = 'MY_QUANDL_API_KEY'

@xw.func
def get_quandl(ticker, start_date=None, end_date=None):
    return quandl.get(ticker, start_date=start_date, end_date=end_date)

And getting the Big Mac Index for the U.S. looks like that:

New built-in Excel formulas: UNIQUE etc.

Dynamic arrays come with a few new built-in Excel functions that are worth mentioning. For example =UNIQUE(A1:A12) returns a list of unique values:

Other new dynamic array formulas worth checking out are:

• FILTER
• RANDARRAY
• SEQUENCE
• SORT
• SORTBY