The sampler itself used R’s sample() function, so I went looking for the corresponding functionality in C++ (assuming Rcpp would have exposed it). And, while much of R’s random number generation is easily accessed, there was no clean way to hook into the C code underlying sample().

Christian Gunning addressed this by contributing a patch to RcppArmadillo which exposes much of the functionality of R’s sample(). We write about it here.

None of the examples in the above link really allow the Rcpp implementation to shine, so I put together an example accept-reject sampler that runs in C++-land thanks to Rcpp. You can see the results with fully working code here.

Altogether, you may not need to use Rcpp if all you want to do is call sample() once. But, if you have to repeatedly make calls to sample(), the code and the documentation exist for you to implement it readily in Rcpp.

1
2
3
4
5
6

install.packages("Rmpi", configure.args="
--with-Rmpi-include=/opt/local/include/openmpi/
--with-Rmpi-libpath=/opt/local/lib/openmpi/
--with-Rmpi-type=OPENMPI
"
)

@ Princeton

If you happen to be setting up Rmpi on one of the TIGRESS systems at Princeton, I suggest a slightly different solution. It has the same effect, but is a bit more self-documenting which allows for users to more obviously edit the setup as their environment changes.

Assuming you use Bash as a shell (and if you don’t know what that means, on the TIGRESS systems, it means you use Bash), add the following to .bashrc.

1
2
3
4
5
6
7
8

## MPI START ##
MPI_ROOT=/usr/local/openmpi/1.4.5/gcc/x86_64/
export MPI_ROOT
LD_LIBRARY_PATH=/usr/local/openmpi/1.4.5/gcc/x86_64/lib64
export LD_LIBRARY_PATH
PATH=/usr/local/openmpi/1.4.5/gcc/x86_64/bin:$PATH
export PATH
## MPI END ##

This sets the values of several environmental variables. This is not a flexible approach. It is requiring you to use OpenMPI version 1.4.5 (compiled withg GCC). Should you want something else, you need to edit these values accordingly.

Motivation

Much of the code that I use for my dissertation project is in C++. I need to generate random variables from a Truncated Normal Distribution and I could not find anything in the Boost libraries or the like. So, I had to implement my own generator. This led me an old article: Robert (1995).

After implementing the sampler described in the paper, I decided to compare its performance to that provided by the truncnorm package in R. While this involved me wrapping some functions around my C++ code and creating some function definitions in R, it wasn’t much additional work.

Benchmark

Below are the results of a benchmark of the performance between my code, rtnRcpp (which uses Rcpp heavily) and truncnorm.

Deviates from a Truncated Normal Distribution with mean 0 and standard deviation 1 are pulled from both packages. Each point on the graph reflects the relative performance of the functions from the two packages, each being called 1,000 times. I ask for three different sample sizes: 10^2, 10^4, and 10^6. And, I consider 6 different intervals. The specific reason for these intervals is motivated by the different cases considered in the algorithm. Beyond that, the details do not matter here.

As a note, I checked to make sure that code for both packages was compiled with the same flags.

As you can see, my code (i.e. rtnRcpp) outperforms truncnorm in every case. The function rtruncnorm() from the truncnorm package is anywhere from 18-80% slower. Sample size doesn’t seem to matter in any set way.

Whatmore, as best I can tell, the authors of the truncnorm package are implementing the same algorithm from Robert (1995), so the performance difference is driven only by the C++ code and not the underlying logic. I have every reason to believe that it is the Rcpp-foundation which leads to my realized speed. The truncnorm package definitely implements a different algorithm. When implemented in straight C, the Robert (1995) algorithm fares less well than it had when implemented in C++. Still, for most regions in the parameter space it is faster or as fast.

Release

I’m happy to share details of my implementation if anyone wants to see it.

I’ve been working on a periodic rebuilding of the Beowulf cluster that we use in the star lab. I stumbled upon a strange message in setting up the OpenMPI implementation of MPI. In case it matters, but I can’t imagine it will, the cluster is being developed on Fedora Core 16.

OpenMPI Warning Message

I kept running into a (to–me cryptic) warning after getting MPI up and running. Whenever an MPI program would run, I’d get output like following printed to the shell:

librdmacm: couldn't read ABI version.
librdmacm: assuming: 4
CMA: unable to get RDMA device list
--------------------------------------------------------------------------
[[14865,1],0]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
Host: someserver

Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------

where I’ve substittued someserver for the actual server name for the purposes of this post. The execution works, but at the least, whatever was going on seemed to be slowing that down. Moreover, I was fairly sure that this output would cause constant confusion for the users once the cluster went live.

UPDATE (Mon Oct 1 2012): It turns out this message isn’t printed in newer versions of OpenMPI (e.g. >=1.6.2) if the hardware is absent. Although I am constrained to using an older version, this is good to note. Thanks to Jeff Squyres for pointing this out.

It turns out that the “problem” and its “solution” are both very simple, but the information I found online wasn’t that good. This is just my humble attempt to improve the signal to noise ratio.

Problem

OpenMPI is searching the hardware on the nodes for InfiniBand, and, upon failing to find any, falls back to standard interfaces. Of course, this is why execution of the =mpirun= command was working.

Solution

We can tell mpirun to never even check for InfiniBand hardware (and there preventing an unsuccessful search) with the inclusion of -mca btl ^openib on the command line. Specifically, a command that originally was submitted as

1

mpirun -np 3 -hostfile ../mpihosts helloworld

should now be

1

mpirun -np 3 -mca btl ^openib -hostfile ../mpihosts helloworld

That is all there is to it.

UPDATE (Mon Oct 1 2012): Initially, I couldn’t find mention of a config file to make this change (in my case) site-wide. Unsurprisingly, this is a result of me not finding it and not the absence of this functionality. Again, thanks to Jeff Squyres for pointing me here which covers this topic. This is now the approach used for this project.

Caveat

This warning can still pop up if you use some other software which wraps OpenMPI and doesn’t have this flag built in (which is not likely). One example is the R package Rmpi. The testing of the package which occurs as install-time in R includes the running of an MPI job. Because this job isn’t run with the flag which disables the search for InfiniBand hardware, you will see the warning print to the shell. However, normal use of Rmpi isn’t affected, since users run R scripts from within the MPI framework and can include this flag.

UPDATE (Mon Oct 1 2012): See the preceding update. This message won’t pop up after using the site-wide config.

Much of the work in my dissertation focuses on generalizing the models used in Political Science for the estimation of ideal points. Despite some shortcomings, there hasn’t been much development of the original model using the canonical roll call voting data. Instead, the innovation has occurred by using novel kinds of data (e.g. speech text or campaign contributions).

In working on some of the shortcomings in how the canonical approach is used in practice, it became necessary to implement mechanics of the Bayesian inference problem—which uses MCMC methods—in C++. Now, since R is a very nice interface relative to C++, the real goal is to just perform the “hard” operations in C++ and keep the rest in R. Enter Rcpp, of course.

Motivation

I was making some speed improvements to my C++ code (which heavily uses Rcpp) and I had to make the decision of how to call the RNG in C++. I could use the old R API with something like Rf_rnorm(0.0,1.0) which would return a double or I could use rnorm(1, 0.0, 1.0)[0] which would also return a double. Now, this second option is not really in the spirit of the syntactic sugar provided by Rcpp. The new API would just as easily let me construct an Rcpp::NumericVector of length N populated with N draws from the Standard Normal distribution with an rnorm(N, 0.0, 1.0) expression. However, short of rewriting a lot of code, I wanted (and needed) to take single draws many times.

The important question to me became, how does performance differ when taking draws one-by-one via the old R API and the new sugary Rcpp API. And, while I was at it, I decided to throw in comparisons to the new Rcpp API used as intended and calling the whole thing from R.

Design

I use the inline, Rcpp, and rbenchmark packages for the test. The first two packages are used to facilitate use of C++ code. The last provides a nice infrastructure to time and compare the evaluation of expressions. Each of 4 approaches is used to take 100,000 draws from a Standard Normal distribution:

1. loop over 100,000 singleton draws from the old R API in C++ via Rcpp (src1)

2. loop over 100,000 draws singleton draws from the new Rcpp API in C++ via Rcpp (src2)

3. a single call to the new Rcpp API asking for a vector of 100,000 draws in C++ via Rcpp (src3)

4. a single call to the standard RNG in R asking for a vector of 100,000 draws

Now, I was certain that (3) would outperform (2), but I didn’t really have any a priori expectations beyond that. You can see my source below for the specifics.

Results

The results are below, lazily formatted as comments in R code. Like we might expect, using the new Rcpp-provided API as intended (Rcpp/Inline/New RNG API/No Loop) is faster than looping over it many times while taking singleton draws (Rcpp/Inline/New RNG API/Loop). Interestingly, though, the new Rcpp-provided API is the fastest method considered. Looping over the old R API (Rcpp/Inline/Old RNG API/Loop) is about 25\% slower. The old R API method and the straight R approach (Straight R) are very close, with the call to the RNG beginning in R (Straight R) being slightly slower. Both of these two approaches outperform the loop over the new Rcpp API (Rcpp/Inline/New RNG API/Loop).

For my purposes, I just needed to see that Rcpp/Inline/Old RNG API/Loop outperformed Rcpp/Inline/New RNG API/Loop, but I couldn’t find any of these benchmarks anywhere online, so I thought I’d put up the full version. I always love it when I come across pareto optimal-type results like this. If you are working in C++, Rcpp/Inline/New RNG API/No Loop not only saves development time (i.e. one line of code, not several), but it saves computational time.

##                              test replications elapsed relative
## 3 Rcpp/Inline/New RNG API/No Loop         1000  16.398 1.000000
## 1    Rcpp/Inline/Old RNG API/Loop         1000  20.191 1.231309
## 4                      Straight R         1000  23.163 1.412550
## 2    Rcpp/Inline/New RNG API/Loop         1000  34.193 2.085193

Edit: Previous results that were posted here and have since been replaced had no optimization performed by the compiler for the three Rcpp-based functions I used. The results that are currently displayed above were compiled with -O3. The rank ordering of the original results does not change. However, some of the discussion of the results reflects new relative performance. Additionally, the number of replications was increased as a modest attempt to make the relative performance results more stable.

Source

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

library(inline)
library(Rcpp)
library(rbenchmark)

src1 <- "
int N_ = as<int>(n) ;
Rcpp::NumericVector draws(N_) ;
RNGScope scope ;
for(int i = 0; i < N_; i++) {
  draws(i) = Rf_rnorm(0.0, 1.0) ;
}
return draws ;
"

src2 <- "
int N_ = as<int>(n) ;
Rcpp::NumericVector draws(N_) ;
RNGScope scope ;
for(int i = 0; i < N_; i++) {
  draws(i) = rnorm(1, 0.0, 1.0)[0] ;
}
return draws ;
"

src3 <- "
int N_ = as<int>(n) ;
Rcpp::NumericVector draws(N_) ;
RNGScope scope ;
draws = rnorm(N_, 0.0, 1.0) ;
return draws ;
"

GetDraws1 <- cxxfunction(signature(n = "integer"),
                         body = src1,
                         plugin = "Rcpp"
                         )
GetDraws2 <- cxxfunction(signature(n = "integer"),
                         body = src2,
                         plugin = "Rcpp"
                         )
GetDraws3 <- cxxfunction(signature(n = "integer"),
                         body = src3,
                         plugin = "Rcpp"
                         )

.n <- 1e5

benchmark("Rcpp/Inline/Old RNG API/Loop" = {set.seed(1); GetDraws1(.n)},
          "Rcpp/Inline/New RNG API/Loop" = {set.seed(1); GetDraws2(.n)},
          "Rcpp/Inline/New RNG API/No Loop" = {set.seed(1); GetDraws3(.n)},
          "R" = {set.seed(1); rnorm(.n)},
          columns=c("test",
            "replications",
            "elapsed",
            "relative"
            ),
          order="relative",
          replications=1000
          )

But who titles a post about running “Translated Exponential Distribution”? Not even me. So, let’s move on.

The second thing I did today was fix some C++ code for random number generation from a Truncated Normal Distribution. The naive Accept-Reject algorithm is great in some cases and terrible in others. None of the R packages that offered an implementation really met my needs, so I implemented the sampler described in Robert (1995) (gated).

Although there is an R interface for my own end-use, everything important happens in C++ via Rcpp. For some parameter settings (see the paper, of course), the algorithm calls for simulating from a Translated Exponential distribution. Personally, I’d never come across this distribution (which maybe should be a point of embarrassment?). Google wasn’t terribly helpful, although I probably could have derived the solution if I felt so inclined. But, that’s more work than the problem really merits given the simplicity of the relationship.

Easy Simulation

Let $x$ be an Exponential random variable such that $x$ has the density $$ f(x) = \exp(-x) * \mathbf{I}_{~x \geq 0}.$$ Then $y = \mu + \frac{x}{\lambda}$ has the density

$$ g(y| \mu, \lambda) = \lambda * \exp(-\lambda(y - \mu)) * \mathbf{I}_{~x \geq \mu} $$

and $y$ is said to be a Translated Exponential random variable with scale parameter $\lambda$ and shift parameter $\mu$.

For simulation purposes, $y = \mu + \frac{x}{\lambda}$ is what matters. First, simulate $x$ from an exponential distribution. Second, transform $x$ to produce $y$. Quite simple as far as these things go.

Below, the objects “A” and “B” must otherwise exist as the output of a ggplot(data) + ... expression.

1
2
3
4
5
6

grid.newpage()
pushViewport(viewport(layout = grid.layout(1, 2)))
vplayout <- function(x, y)
viewport(layout.pos.row = x, layout.pos.col = y)
print(A, vp = vplayout(1, 1))
print(B, vp = vplayout(1, 2))

The reason I switched was that the old templater was so inelegant and clunky that I ended up not updating content as often as I should have. This switch should remove some of that friction.