R on Yoke Keong

Running Apache Spark with sparklyr and R in Windows

Wed, 06 Jul 2016 17:00:50 +0000

RStudio recently released the sparklyr package that allows users to connect to Apache Spark instances from R. In addition, this package offers dplyr integration, allowing you to utilize Spark as you use dplyr functions like filter and select, which is very convenient. The package will also assist you in downloading and installing Apache Spark if it is a fresh install. This post covers the local install of Apache Spark via sparklyr and RStudio in Windows 10.

Downloading R packages securely

Tue, 25 Aug 2015 11:00:52 +0000

JJ Allaire from RStudio wrote a step-by-step guide on how to configure secured downloads of R packages. Secured downloading of R packages (via HTTPS connection) ensures that you get your packages from legitimate, trusted sources.

Getting session information in R

Tue, 30 Jun 2015 11:00:15 +0000

When troubleshooting R bugs or asking for assistance in mailing lists and sites like StackOverflow, it is good to review or present information about your system and packages loaded.

I much prefer the session_info() function from the devtools package over the default sessionInfo() function as it’s output is not only more readable, it also provides useful information like timezone and additional packages (non-base) loaded at the time.

Assuming you have the devtools packages already installed, you can invoke the function in one line:

Updating packages after R upgrade

Tue, 26 May 2015 11:00:50 +0000

Note to self: After upgrading R (or Revolution R Open) on Windows, run the following command to update the packages at one go.

1update.packages(checkBuilt = TRUE, ask = FALSE)

Drawing map shapes with rgdal

Tue, 05 May 2015 11:00:06 +0000

In this post, let us explore the R package rgdal for map shape plotting. We shall attempt to plot the map of Singapore and display major road networks in Singapore.

Pre-requisites

To get the data you need, you can go to a GIS provider. In this post, we shall be using diva-gis.

Steps:

1. Go to: http://www.diva-gis.org/gdata

2. Select ‘Spatial Data Download’

3. Select Country = ‘Singapore’

4. Select Subject = ‘Administrative areas (GADM)’

Reading tabular data with readr package

Tue, 14 Apr 2015 11:00:57 +0000

Recently, Hadley Wickham introduced a new package to read tabular data (such as CSV), lines and entire files.

Advantages include:

1. Helpful defaults over base R read.csv such as: Characters are never automatically converted to factors and row names are never set.

2. Faster reads.

3. When reading large files, a progress bar is displayed.

4. A very useful problems() function that allows you to zoom into rows of data that readr has issues loading into for e.g. expected integer for this column but actual data is text.

Reading excel files with readxl R package

Tue, 31 Mar 2015 11:00:05 +0000

Recently, Hadley Wickham introduced a new package to read Excel files (XLS, XLSX). The main advantage is that no external dependencies is required for readxl. (xlsx package requires Java Runtime to be installed)

With xlsx:

1library(xlsx)
2library(httr)
3url <- "https://rawgit.com/yoke2/dsxref/master/iris.xlsx"
4GET(url, write_disk("iris.xlsx", overwrite=TRUE))
5iris <- read.xlsx("iris.xlsx", sheetIndex=1, header=TRUE)
6head(iris, 3)

1## NA. Sepal.Length Sepal.Width Petal.Length Petal.Width Species
2## 1 1 5.1 3.5 1.4 0.2 setosa
3## 2 2 4.9 3.0 1.4 0.2 setosa
4## 3 3 4.7 3.2 1.3 0.2 setosa

With readxl:

Download files over HTTPS in R with httr

Tue, 10 Mar 2015 11:00:52 +0000

To download a file over HTTP connection, we normally use download.file command in R, for example:

1url = "http://vincentarelbundock.github.io/Rdatasets/csv/datasets/iris.csv"
2download.file(url, "iris.csv", quiet=TRUE)

For HTTPS connections, download.file may give you some issues. In situations like this you can consider using the httr package to download files:

1library(httr)
2url <- "https://rawgit.com/yoke2/dsxref/master/iris.xlsx"
3GET(url, write_disk("iris.xlsx", overwrite=TRUE))

Data Science Learning - A Cross Reference

Tue, 17 Feb 2015 11:00:12 +0000

While learning data science, I’ve discovered that it is very useful to think of the data science processing as a “pipeline” i.e. a series of actions in a process. Along this pipeline, you will be tackling lots of “How do I…” questions like “How do I remove NA values?” and “How do I create N-grams?”

Furthermore, given the many data science tools and languages available online, you will most likely ask the same questions again when you are learning how to perform data science tasks in another language. While Google Search and Stack Overflow/Stack Exchange comes in very handy when searching for answers, I wanted some structure - a collection of sorts - to these questions and have working examples in different language implementations.

A virtual environment for data science

Tue, 13 Jan 2015 11:00:23 +0000

I wanted to conveniently use data science tools without the hassle of installing the required languages and packages, while benefiting from the strengths of the Linux command line tools. There is a pre-packaged VM called the Data Science Toolbox that fills this need.

It comes with R and Python installed, along with the respective popular data analysis packages for R and Python. You will be able to install the VM successfully by following the instructions on the website, including installation of pre-requisites like VirtualBox and Vagrant.

Speeding up R

Tue, 16 Dec 2014 11:00:38 +0000

When processing large datasets, some R commands operations requiring linear algebra libraries may run very slowly. As such you might want to try Revolution R Open, an enhanced R distribution using multi-threaded libraries. You can see some benchmarks here and here.

Exploring Focal Length with Exiftool and R

Thu, 16 Oct 2014 12:00:36 +0000

Picture Credits: Pixabay

A good way to understand your shooting style and guide your future camera equipment buying decisions will be to discover your frequently used focal lengths.

Focal Lengths can be extract from photos that have EXIF data, which in short refers to data on how these photos are taken. You can find out more from an introductory article here. In this post, we are going to explore focal length usage with the flexible Exiftool and R.