Don't standardize interaction/moderator effects in multiple regression

This is another quick blog entry related to a query I had today.

In my moderated multiple regression workshop a while back I wrote (on slide 15) "don't use standardized regression coefficients". In the talk itself I briefly mentioned why and directed people to the relevant section of the (excellent) explanation by Kris Preacher:

A primer on interaction effects in multiple linear regression.

I'd like to clarify why standardization is a particularly bad idea in this case. My dislike of 'standardization' is fairly well known, and it goes without saying (I hope) that one reason not to use standardized regression coefficient relates to this. I hope to write about this in detail soon (but check out my BJP article on effect size for further details).

However, there are additional reasons why standardizing predictors will cause trouble in moderated  multiple regression. Standardization involves centering the predictors and scaling them terms of their sample SDs. Centering is very often a useful thing to do in moderated multiple regression. However,  statistics packages such as SPSS will standardize all the predictors - including the product terms -  in moderated multiple regression. This is because they have no way of knowing that the product term is not a 'regular' predictor (similarly if anyone were foolish enough to do a stepwise regression ... the software would not know to keep in X1 and X2 for each X1.X2 product term). This means that the X1.X2 product term will be standardized along with X1 and X2 rather than being computed (correctly) as the product of the two standardized predictors (i.e., Zx1 and Zx2 multiplied together).

That's clearly a problem (the t test should be OK but the value of the coefficient and simple slopes will be wrong). However, the clincher is that even if the correct standardization is carried out (e.g., computing the standardized predictors yourself and then taking the product of the relevant standardized predictors and entering it into the regression) the standard errors will be incorrect (which is problematic for constructing confidence and prediction intervals).

In summary, you can't rely on the software to get the standardization correct so use the unstandardized regression coefficients and standard errors! (Also, did I mention that standardized coefficients are generally a bad idea anyway?).

R resources for psychologists

As a quick follow-up to my previous post, I'll quickly note some R links specifically aimed at psychologists. Jonathan Baron and Yuelin Li have a excellent set of notes that cover ground from R basics to ANOVA (including repeated measures ANOVA) and beyond. Jonathan Baron also maintains a general set of R links. William Revelle's R guide for psychological research is also very good and has more emphasis on psychometrics than Baron and Li.

Pilgrim's blog is an R resource for the exercises in the classic Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences text.

Simulating data for inquiry based learning

I've just presented a talk on at the Statistics for Psychology Students workshop for the HEA Psychology Network in York. Richard Rowe (Psychology, Sheffield) gave an interesting talk on teaching statistics via inquiry based learning. Part of the work involved using getting students to generate their own research questions in a tutorial and then analyzing data addressing these questions in a follow-up tutorial. A rather clever idea he reported was to generate suitable data via simulation in STATA. The students were first or second year undergraduates and so the experimental designs they came up with were constrained to simple cases (e.g., t tests, correlations and chi-square for the first years; one-way ANOVA with 3 levels for the second years). This meant that whoever led the tutorial (e.g., a Ph.D. student) could generate tailored data to match the students' research designs.

The novel and clever aspect of this isn't simulating real data sets or getting students to come up with their own research ideas, but combining the two. This means that you can run these sessions without getting students to collect real data. (There is a place for that too, but collecting real data has massive overheads in terms of student time, staff time, ethics approval and so forth). Furthermore, students can legitimately come up with ideas for studies that can't be run by undergraduates for ethical, resource or other reasons - for example using clinical populations). I can also see other uses for this. For example, a supervisor could simulate data for a final year project and a student could use the simulated data as a dry run for the real analysis. (The real data will usually be much messier, but I think that getting familiar with the statistical software and analyses to be used could be useful for some students).

A few members of the audience asked questions about STATA (which I'm not familar with) and I pointed out that you could do the same things in R for free. As I understand it STATA allows you to simulate data with a specified covariance matrix fairly easily. I'm sure this can be done in R too, but I'm still learning how to use R and tracking down the right package and commands would have taken a bit of time. In any case, for this to be useful it needs to be very simple and easy to run by people with relatively basic statistical computing skills (and who have never used R before). So I set myself a challenge of writing code that should run with functions from the R base package, be trivial to edit and generate usable data for two common analyses: the independent t test and a bivariate correlation.

It took me about 2 or 3 minutes to write the independent t test code. (This may sound impressive but a competent R programmer could probably do it in under a minute). It took a little longer to comment it and work out how to write it to a tab delimited text file. It took me about 5 or 10 minutes to do the bivariate correlation and a little longer to fix a rather stupid error I'd made. All the code is rather clumsily written (and in some cases I've deliberately separated out steps to make it easier for someone unfamiliar with R to follow). The correlation solution (in particular) isn't very good and specifying a covariance matrix to constrain a simulation would be much more satisfactory. (I should also note that it uses a trick I picked up some time ago from Paul Barrett's web site in his article on Correlation attenuation due to Likert categorization.) The key point is that it took longer to describe here than it took to write. I'm sure better solutions exist in R, but these ones work and could easily be extended to paired t tests or one-way ANOVA. (I'm not sure about chi-square. I could write something, but it might be a lot easier just to specify a 2x2 or 3x2 contingency table by hand).

Running the R code is also easy. R runs on PC, Linux and Mac OS X, but if you don't fancy installing it (for some bizarre reason) it can also be run from a web server (though it won't be able to write to a file). To run the code you just paste it into the R console and hit return. To tailor the code just read the comments and tweak the parameter values before pasting.

If you want a copy to play with just email me.

For anyone who wants to do some simulation in R there are lots of resources around (try googling the obvious keywords), but Andrew Gelman and Jennifer Hill's book Data Analysis Using Regression and Multilevel/Hierarchical Models (Analytical Methods for Social Research) is an impressive book for a serious take on using simulation (although the stance is Bayesian this doesn't get in the way - at least for the sections I've read; I'm about 40% through the book). 

Postscript. All the talks were video recorded and Anne Trapp threatened to put video podcasts - vodcasts? - on the Psychology Network site at some point in the future - including my talk (Effect size: why what we teach psychology students is wrong). These will be worth looking out for just to see Andy Field's talk. (Andy claims he looks particularly ridiculous when video recorded. My advice: just don't ever look at the recording. Ever.) I'll write something about effect size in the near future - though much of the talk was based on my forthcoming BJP paper on effect size.