Skip to main content

excel: my first use of power query (and i love it)

let’s face it. if you’re a techie and you don’t use excel, you are not peeking out your geeking out. :o) i use excel for a number of different things. it’s a really powerful program which can handle doing much more than figuring out how much i’ve spent on lunch over the last three months.

at teched, i got my first taste of power query during some of the hands-on-labs (available online for free now.) something came up recently that gave me a chance to explore it a bit more to see its value. let’s explore a scenario where your organization is absorbed or is absorbing another organization. after a domain migration, human resources decides they want to start over with new employee IDs.

 

SCENARIO

HR provides you with a file claiming it has all the information you need. (and clearly, you’ve no reason to doubt their claim.) upon examining you file, you notice that the only thing in the file is a column with the old employee ID and a column with the new employee ID.

drawing from your history, you know that employeeID is not an indexed attribute in active directory and would take quite a length of time to query each user by their old employeeID and write their new one. you’re also short on time to get any changes to the AD schema pushed through.

knowing you’ll need it, you create your own export of user ID and employee ID from active directory. game on.

 

THE VLOOKUP WAY

the way i would have approached this in the past (and did to be honest until my colleague encouraged me to use power query) is to create two sheets in excel. one sheet would contain the information from HR, and the other would contain the information from AD. from the HR sheet, you could run a vlookup formula in a new column, pulling the user IDs that match from AD.

 

THE POWER QUERY WAY

i’m about to demonstrate is magic, my friends. magic known as PFM… the purest kind available. i’m going to take one file that looks with just the old id and the new id and merge it with another that contains the old id and the user id ending up with a new dataset that contains the old id, the new id, and the user id. there’s also a little wrinkle here. the new id value they want starts with a leading zero. excel loves converting them which basically switches it to a number, removing the leading zero. no problem.

here’s a sample of how the files look:

SNAGHTML24c0fb47

  1. open excel (DUH) and start with a blank worksheet.
  2. if you haven’t downloaded power query, you will need to do that first.
  3. switch to the power query tab.
  4. select from file > from csv.
  5. open the first .csv file.

everything looks good except that leading 0 problem. hell, even the first row header was automatically applied. the leading 0 is a non-issue since it can be changed as an applied step!

  1. change the query name to emplIDs.
  2. click the new id column.
  3. right-click the new id column and choose change type > text.
  4. make sure the old id column stays as number. if not, change the type to number.
  5. under load settings, choose load to data model only.
  6. click apply & close.

image

you can’t see it in the worksheet, but have faith in magic that it’s all there (or use the peek by hovering your mouse over the query.)

image

let’s work on getting the merged results from both files into a worksheet now that we have a dataset with which we can combine.

  1. on the power query tab, choose from file > from csv.
  2. open the second file.
  3. name the query userIDs.
  4. choose merge queries from the ribbon bar.
  5. drop down the query selection and choose emplIDs.
  6. select the old id column in both tables and click OK.

a new column now shows up in the query with a peculiar little symbol image and values that state Table. we need to indicate what we want to pull back from the emplIDs data model.

  1. click the symbol of confusion.
  2. select the column new id and click OK.
  3. rename the new column to new id.
  4. click apply & close.
image

the result should be a beautifully merged set of data!

image

Comments

Popular posts from this blog

how to retrieve your ip address with powershell...

update: this is how it’s performed in powershell v3 as demonstrated here.(get-netadapter | get-netipaddress | ? addressfamily -eq'IPv4').ipaddress update: this is by far the easiest.PS C:\temp> (gwmi Win32_NetworkAdapterConfiguration | ? { $_.IPAddress -ne $null }).ipaddress
192.168.1.101
are you laughing yet?  i know you probably find this topic amusing.  it's really interesting though.  whenever you get over it, i'll do this in the standard cmd.exe interpreter and then in powershell to show you what kind of coolness powershell does.done?  okay, good.  this is an interpretation of a demo that bob wells did at our smug meeting.  hope you like it.i should tell you, it's not as simple as the title would lead you to believe.  i like doing that little slight-of-hand thing since it gives the impression that i'm painting a very easy target on my back for your criticism (though it's probably true in other ways)!  the idea is that we want to retrieve just the ip ad…

understanding the “ad op master is inconsistent” alert

i use the term “understanding” loosely.  this is by far no definitive guide on this particular alert, just a few things i have picked up in my attempt to understand it.let’s look at the context of the alert:The Domain Controller's Op Master is inconsitent. See additional alerts for details.
first of all, it gives very little information.  the only particularly useful detail is that it indicates which server is having the issue.  other than that, just a spelling error as there are no additional critical alerts to look at for details.this rule, as you know, comes from a sealed mp.  therefore, we can’t modify anything in it except the overrides.  the couple i’ve tinkered with are:interval (sec) log success event to begin with, interval (sec) is just set way too high.  the default is 60 seconds.  why on earth would anyone want to know that your op master consistency may be off, every minute?  actually, i could think of a few reasons, but really, it’s overkill.  the way the script works…

sccm: content hash fails to match

back in 2008, I wrote up a little thing about how distribution manager fails to send a package to a distribution point. even though a lot of what I wrote that for was the failure of packages to get delivered to child sites, the result was pretty much the same. when the client tries to run the advertisement with an old package, the result was a failure because of content mismatch.I went through an ordeal recently capturing these exact kinds of failures and corrected quite a number of problems with these packages. the resulting blog post is my effort to capture how these problems were resolved. if nothing else, it's a basic checklist of things you can use.DETECTIONstatus messagestake a look at your status messages. this has to be the easiest way to determine where these problems exist. unfortunately, it requires that a client is already experiencing problems. there are client logs you can examine as well such as cas, but I wasn't even sure I was going to have enough material to …