R on DataBaselines

DAT102 Data Science Capstone

Thu, 09 Nov 2017 00:00:00 +0000

With both pride and relief – I get to share news of completing the Capstone course for the Microsoft Professional Program in Data Science. Over the past six months, I’ve been systematically working through the nine courses leading up to this final 10th course in the series. You can look on this index page to see observations for the other courses. This includes some general advice for those considering taking these courses.

The blog entry you are reading now is specific to the Capstone course. It explains the hoops you need to jump through, the time you will likely need to complete, and a variety of other observations about the Capstone.

Using R to Evaluate College Scorecard

Fri, 03 Nov 2017 00:00:00 +0000

Our company acquired a data file containing over 15,000 rows and 300 columns. We are trying to identify patterns in the data. Where do we begin evaluating such a large dataset? Would Using R be helpful?

Let me answer that question by taking a look at data from the U.S. Department of Education known as the College Scorecard. This organization compiles a wide variety of measures for all colleges and universities in the United States. They track over 300 measures such as:

Average SAT scores for incoming students,
Percentage of students whose parents completed high school, and
Whether the Institution is private or publicly owned.

I originally evaluated this data as part of a capstone course in the Microsoft Professional Program for Data Science. In that class, the overall goal was to identify a subset of these 300 measures that were most useful in predicting a single measure – the average income of graduates. Admittedly, it’s a bit overwhelming to know where to start resolving this sort of problem. You have intuitive hunches that SAT scores or faculty salary might be correlated to the future income of graduates. But how do you systematically evaluate a data set this large?

DAT213 - Analyzing Big Data with Microsoft R Server

Tue, 25 Jul 2017 00:00:00 +0000

This course teaches exploratory data analysis skills using the Microsoft R Server implementation known as RevoScaleR. This product is in most ways functionally equivalent to the open source CRAN-R. RevoScaleR offers three significant benefits over its open source brother: the ability to run analyses in parallel across different servers, the ability to “chunk” data for evaluation and bypass the in-memory limitation of R, and the ability to read more natively from data sources like SQL Server, Hadoop, and Spark. This course explains these benefits and allows a new user to become familiar with the RevoScaleR tool.

Analyzing Big Data with Microsoft R Server
#

The course is divided into 4 segments:

DAT209 - Programming R

Mon, 10 Jul 2017 00:00:00 +0000

Programming R for Data Science is taught by Anders Stockmarr (on the faculty of Technical University of Denmark.) For US audiences, his accent requires some getting used to. He places emphasis on unexpected syllables and has a unique way of pronouncing many things. I found it helpful to use headphones and to adjust the playback speed of the recordings. It is worth making the effort to understand Dr. Stockmarr because he has put together a course with a lot of substance, using a tight script and backed up by supporting exercises.

Programming R Course Highlights
#

I genuinely enjoyed this course, it goes a lot deeper than the introductory course in R taken earlier in the MPP sequence. For the first course, I used RStudio to experiment. With this course, I wanted to use the Visual Studio version of R to work the exercises and labs. The R Tools for Visual Studio (https://www.visualstudio.com/vs/rtvs/) required some fiddling to get installed, but were stable and had nice IDE features I’ve become used to with VS. Becoming familiar with R Tools for Visual Studio at this point will prepare you for taking DAT213 “Analyzing Big Data in MS R Server” which is the logical follow-on course in the MPP Data Science sequence.

DAT204 - Intro to R for Data Science

Thu, 15 Jun 2017 00:00:00 +0000

As a developer, I’m drawn to terse/concise languages that are purpose-built for an objective. Regular expressions are a prime example. There is something beautiful about expressing things in few words (something I try to do in blogging with only partial success!) In this context, I was eagerly anticipating “Intro to R for Data Science.” This course (and this language) did not disappoint.

Before going further, I should note something: Within the Microsoft Professional Program for Data Science, it is the student’s discretion to take a Python or an R track. Your decision will be shaped by whether you have prior familiarity with one of those environments, and whether you want to reinforce what you already know or venture into a new tool. Your choice of this class will logically dictate the 2nd “advanced” course required later in the MPP track.