Skip to main content

excel: my first use of power query (and i love it)

let’s face it. if you’re a techie and you don’t use excel, you are not peeking out your geeking out. :o) i use excel for a number of different things. it’s a really powerful program which can handle doing much more than figuring out how much i’ve spent on lunch over the last three months.

at teched, i got my first taste of power query during some of the hands-on-labs (available online for free now.) something came up recently that gave me a chance to explore it a bit more to see its value. let’s explore a scenario where your organization is absorbed or is absorbing another organization. after a domain migration, human resources decides they want to start over with new employee IDs.

 

SCENARIO

HR provides you with a file claiming it has all the information you need. (and clearly, you’ve no reason to doubt their claim.) upon examining you file, you notice that the only thing in the file is a column with the old employee ID and a column with the new employee ID.

drawing from your history, you know that employeeID is not an indexed attribute in active directory and would take quite a length of time to query each user by their old employeeID and write their new one. you’re also short on time to get any changes to the AD schema pushed through.

knowing you’ll need it, you create your own export of user ID and employee ID from active directory. game on.

 

THE VLOOKUP WAY

the way i would have approached this in the past (and did to be honest until my colleague encouraged me to use power query) is to create two sheets in excel. one sheet would contain the information from HR, and the other would contain the information from AD. from the HR sheet, you could run a vlookup formula in a new column, pulling the user IDs that match from AD.

 

THE POWER QUERY WAY

i’m about to demonstrate is magic, my friends. magic known as PFM… the purest kind available. i’m going to take one file that looks with just the old id and the new id and merge it with another that contains the old id and the user id ending up with a new dataset that contains the old id, the new id, and the user id. there’s also a little wrinkle here. the new id value they want starts with a leading zero. excel loves converting them which basically switches it to a number, removing the leading zero. no problem.

here’s a sample of how the files look:

SNAGHTML24c0fb47

  1. open excel (DUH) and start with a blank worksheet.
  2. if you haven’t downloaded power query, you will need to do that first.
  3. switch to the power query tab.
  4. select from file > from csv.
  5. open the first .csv file.

everything looks good except that leading 0 problem. hell, even the first row header was automatically applied. the leading 0 is a non-issue since it can be changed as an applied step!

  1. change the query name to emplIDs.
  2. click the new id column.
  3. right-click the new id column and choose change type > text.
  4. make sure the old id column stays as number. if not, change the type to number.
  5. under load settings, choose load to data model only.
  6. click apply & close.

image

you can’t see it in the worksheet, but have faith in magic that it’s all there (or use the peek by hovering your mouse over the query.)

image

let’s work on getting the merged results from both files into a worksheet now that we have a dataset with which we can combine.

  1. on the power query tab, choose from file > from csv.
  2. open the second file.
  3. name the query userIDs.
  4. choose merge queries from the ribbon bar.
  5. drop down the query selection and choose emplIDs.
  6. select the old id column in both tables and click OK.

a new column now shows up in the query with a peculiar little symbol image and values that state Table. we need to indicate what we want to pull back from the emplIDs data model.

  1. click the symbol of confusion.
  2. select the column new id and click OK.
  3. rename the new column to new id.
  4. click apply & close.
image

the result should be a beautifully merged set of data!

image

Comments

Popular posts from this blog

using preloadpkgonsite.exe to stage compressed copies to child site distribution points

UPDATE: john marcum sent me a kind email to let me know about a problem he ran into with preloadpkgonsite.exe in the new SCCM Toolkit V2 where under certain conditions, packages will not uncompress.  if you are using the v2 toolkit, PLEASE read this blog post before proceeding.   here’s a scenario that came up on the mssms@lists.myitforum.com mailing list. when confronted with a situation of large packages and wan links, it’s generally best to get the data to the other location without going over the wire. in this case, 75gb. :/ the “how” you get the files there is really not the most important thing to worry about. once they’re there and moved to the appropriate location, preloadpkgonsite.exe is required to install the compressed source files. once done, a status message goes back to the parent server which should stop the upstream server from copying the package source files over the wan to the child site. anyway, if it’s a relatively small amount of packages, you can

How to Identify Applications Using Your Domain Controller

Problem Everyone has been through it. We've all had to retire or replace a domain controller at some point in our checkered collective experiences. While AD provides very intelligent high availability, some applications are just plain dumb. They do not observe site awareness or participate in locating a domain controller. All they want is the name or IP of one domain controller which gets hardcoded in a configuration file somewhere, deeply embedded in some file folder or setting that you are never going to find. How do you look at a DC and decide which applications might be doing it? Packet trace? Logs? Shut it down and wait for screaming? It seems very tedious and nearly impossible. Potential Solution Obviously I wouldn't even bother posting this if I hadn't run across something interesting. :) I ran across something in draftcalled Domain Controller Isolation. Since it's in draft, I don't know that it's published yet. HOWEVER, the concept is based off

sccm: content hash fails to match

back in 2008, I wrote up a little thing about how distribution manager fails to send a package to a distribution point . even though a lot of what I wrote that for was the failure of packages to get delivered to child sites, the result was pretty much the same. when the client tries to run the advertisement with an old package, the result was a failure because of content mismatch. I went through an ordeal recently capturing these exact kinds of failures and corrected quite a number of problems with these packages. the resulting blog post is my effort to capture how these problems were resolved. if nothing else, it's a basic checklist of things you can use.   DETECTION status messages take a look at your status messages. this has to be the easiest way to determine where these problems exist. unfortunately, it requires that a client is already experiencing problems. there are client logs you can examine as well such as cas, but I wasn't even sure I was going to have enough m