Recently, I needed to detect whether a matrix consists entirely of missing values. I wrote the following module: proc iml; /** Module to detect whether all elements of a matrix are missing values. Works for both numeric and character matrices. Version 1 (not optimal) **/ start isMissing(x); if type(x)='C' then

# Author

There are three kinds of programming errors: parse-time errors, run-time errors, and logical errors. It doesn't matter what language you are using (SAS/IML, MATLAB, R, C/C++, Java,....), these errors creep up everywhere. Two of these errors cause a program to report an error, whereas the third is more insidious because

Both covariance matrices and correlation matrices are used frequently in multivariate statistics. You can easily compute covariance and correlation matrices from data by using SAS software. However, sometimes you are given a covariance matrix, but your numerical technique requires a correlation matrix. Other times you are given a correlation matrix,

Sample covariance matrices and correlation matrices are used frequently in multivariate statistics. This post shows how to compute these matrices in SAS and use them in a SAS/IML program. There are two ways to compute these matrices: Compute the covariance and correlation with PROC CORR and read the results into

I enjoy reading about the Le Monde puzzles (and other topics!) at Christian Robert's blog. Recently he asked how to convert a number with s digits into a numerical vector where each element of the vector contains the corresponding digit (by place value). For example, if the number is 4321,

The SAS/IML language enables you to perform matrix-vector computations. However, it also provides a convenient "shorthand notation" that enables you to perform elementwise operation on rows or columns in a natural way. You might know that the SAS/IML language supports subscript reduction operators to compute basic rowwise or columnwise quantities.

My last post was a criticism of a statistical graph that appeared in Bloomberg Businessweek. Criticism is easy. Analysis is harder. In this post I re-analyze the data to present two graphics that I think should have replaced the one graphic in Businessweek. You can download the SAS program that

Recently I read a blog that advertised a data visualization competition. Under the heading "What Are We Looking For?" is a link to a 2007 Bloomberg Businessweek graph that visualizes how participation in online social media activities vary across age groups. The graph is reproduced below at a smaller scale:

Errors. We all make them. After all, “to err is human.” Or, as programmers often say, “To err is human, but to really foul things up requires a computer” (Farmer’s Almanac, 1978). This post describes how to interpret error messages from PROC IML that appear in the SAS log. The

I am thankful to be a statistical programmer. When I wake up in the morning, I am eager to start my day. I love statistics, programming, and working at SAS, and I write my blog to share that joy. This a Golden Age for statistical programmers because theoretical ideas and

I give many presentations and workshops on how to use SAS/IML Studio, and more than once I have been asked about how to launch the program. Sometimes the inquiry hints at mild frustration, such as last week's "How do I RUN the $%#@# THING!!!!" The email I got this week

In a previous post, I used statistical data analysis to estimate the probability that my grocery bill is a whole-dollar amount such as $86.00 or $103.00. I used three weeks' grocery receipts to show that the last two digits of prices on items that I buy are not uniformly distributed.

In a previous post, I discussed computing regression coefficients in different polynomial bases and showed how the coefficients change when you change the basis functions. In particular, I showed how to convert the coefficients computed in one basis to coefficients computed with respect to a different basis. It turns out

I am pleased to announce that the fine folks at SAS Press have made Chapter 2 of my book, Statistical Programming with SAS/IML Software available as a free PDF document. The chapter is titled "Getting Started with the SAS/IML Matrix Programming Language," and it features More than 60 fully functional

The other day I was at the grocery store buying a week's worth of groceries. When the cashier, Kurt (not his real name), totaled my bill, he announced, "That'll be ninety-six dollars, even." "Even?" I asked incredulously. "You mean no cents?" "Yup," he replied. "It happens." "Wow," I said, with

Chris started a tradition for SAS Press authors to post a photo of themselves with their new book. Thanks to everyone who helped with the production of Statistical Programming with SAS/IML Software.

Suppose that you compute the coefficients of a polynomial regression by using a certain set of polynomial effects and that I compute coefficients for a different set of polynomial effects. Can I use my coefficients to find your coefficients? The answer is yes, and this article explains how. Standard Polynomial

I just got back from a great conference in San Diego at the 2010 meeting of the Western Users of SAS Software (WUSS) where I gave several presentations on PROC IML and SAS/IML Studio. If you didn't make it to San Diego, you can still read my 2010 paper on

Sampling with replacement is a useful technique for simulations and for resampling from data. Over at the SAS/IML Discussion Forum, there was a recent question about how to use SAS/IML software to sample with replacement from a set of events. I have previously blogged about efficient sampling, but this topic

This post is about an estimate, but not the statistical kind. It also provides yet another example in which the arithmetic mean is not the appropriate measure for a computation. First, some background. Last week I read a blog post by Peter Flom that reminded me that it is wrong

Today I'm in San Diego at the 2010 meeting of the Western Users of SAS Software (WUSS). I am giving several presentations on SAS/IML and SAS/IML Studio: A tutorial workshop on SAS/IML Studio for the SAS/STAT User. The material in this tutorial is a small sampling of Chapters 4–11 of

In this blog and in the book Statistical Programming with SAS/IML Software, I present tips and techniques for writing efficient SAS/IML programs for data analysis, simulation, matrix computations, and other topics of interest to statistical programmers. When I was writing my book, one of the reviewers commented that he wasn’t

How can you change a programming trick into a programming treat? Try this algorithm: If you develop a clever snippet of code, squirrel it away. This snippet is a "trick." If you use the trick a second time, copy and modify the code. The trick has become a "treat." If

The SAS/IML language provides the QUAD function for evaluating one-dimensional integrals. You can also use the QUAD function to compute a double integral as an iterated integral. A One-Dimensional Integration Suppose you want to evaluate the following integral: To evaluate this integral in the SAS/IML language: Define a function module

I was recently asked how to create a tridiagonal matrix in SAS/IML software. For example, how can you easily specify the following symmetric tridiagonal matrix without typing all of the zeros? proc iml; m = {1 6 0 0 0, 6 2 7 0 0, 0 7 3 8 0,

In a previous post, I discussed how to use the LOC function to eliminate loops over observations. Dale McLerran chimed in to remind me that another way to improve efficiency is to use subscript reduction operators. I ended my previous post by issuing a challenge: can you write an efficient

Today is World Statistics Day, an event set up to "highlight the role of official statistics and the many achievements of the national statistical system." I want to commemorate World Statistics Day by celebrating the role of the US government in data collection and dissemination. Data analysis begins with data.

The IMLPlus language has been available to SAS customers since 2002, but there are still many people who have never heard of it. What is IMLPlus? The documentation SAS/IML Studio for SAS/STAT Users says this about IMLPlus: The programming language in SAS/IML Studio, which is called IMLPlus, is an enhanced

Have you ever been stuck while trying to solve a scrambled-word puzzle? You stare and stare at the letters, but no word reveals itself? You are stumped. Stymied. I hope you didn't get stumped on the word puzzle I posted as an anniversary present for my wife. She breezed through

A few people asked me to explain the significance of the cartoon in the scrambled-word puzzle that I posted as an anniversary present for my wife. The cartoon refers to a famous experiment devised by Sir Ronald A. Fisher.