Stata: Dealing with Date variables

Dealing with date variables in dataset requires additional steps. If not done correctly, your data format may not be consistent and you will get your value wrong. Here are some tips related to date variables.

Generating date variables

If you import your data from excel, your date variables may be treated as 1) string variables, or 2) integer (less likely). An example of date variable in string format is: 1 Jan 1960. If in integer format, it would be like 17869. These two formats can be easily converted into date format.

String format

If your value is like 1 Jan 1960, you can use the following code:

gen newvar = date(oldvar, "DMY”)

After running this line, you will see that your value would be like 0, or 1046, because these values are stored in integer format. The value represents the days before or after 1 Jan, 1960. Positive value means after 1 Jan, 1960 and negative value means prior to the date. This format is commonly used in different dataset. Don’t be suprised and we will convert it to date format later.

Integer format

If your values are in integer format, you need the following line to convert it to date format.

Simple version

format newvar %d

A bit complicated version:

format %tdnn/dd/CCYY newvar


You can choose either one.

Date comparisons

Once you convert to date format, it is very easy to compare two dates. If you want to see how many days between two date variables, simply use gen command and use one minus another.

Add or minus from a date

If you want to add, for example, 120 days to your date variable or minus 120 days, it is very similar to date comparisons. Simply generate another variable, which equals to your date variable add or minus 120 days.

Comparison with a specific date

If you want to compare a date varialbe with a specific date, such as 1 Jan 2000, you can use the following code:

gen newvar = (mdy(1,1,2000) - oldvar)

The code above will show you how many days before or after 1 Jan, 2000. If your date varialbe is birthday, you can divide the value with 365.25 and get how old your participants are on 1 Jan, 2000.

Further reading

Using dates in Stata http://www.ats.ucla.edu/stat/stata/modules/dates.htm

Stata: Count groups by individuals

One friend asked me the following question:

How can I transform the following format into:

id level
1 A
1 A
1 B
2 A
2 B
3 B






this one?

id level #ofA #ofB
1 A 2 1
1 A 2 1
1 B 2 1
2 A 1 1
2 B 1 1
3 B 0 1

Well, I do not think there is one command for this task. This is not very difficult if there are only two groups to count.

The way I achieve this task is:

use "http://images.researcher20.com/stata_group/stata_group.dta", clear
egen acount = group(level)
gsort +id +acount
by id: egen acount2 = count(acount) if acount==1
bys id: replace acount2 = acount2[_n-1] if acount2==.
replace acount2=0 if acount2==.
bys id: egen bcount2 = count(acount) if acount==2
gsort +id -level
by id: replace bcount2 = bcount2[_n-1] if bcount2==.
replace bcount2=0 if bcount2==. 

Level variable is a string variable, so I use egen to get a group id. If you are interested in learning more details, you can check my previous post:Stata: Create id by group.

After creating a new group id, I sort id and level. How do I count how many As and Bs? I count # of As using egen, but  you may notice that if the value is B, # of A would be missing. So my next step is to fill up this missing with the value of previous record. This is why I sorted data at the beginning.

If there is no A, then replace the value from missing to zero.

The way I count # of Bs is similar. The only difference is sorting.

You may be curious: how about if I have more than three groups? Well, my code only works for two groups, and I have not found a way to count three groups by individuals.

If you have tips or code to achieve the task, please let me know!

Update
2012.4.27:

One friend shared with me her code:
foreach i in A B C D E F G H I J K L N P Q R S T U V W X Y Z {
bys id: egen nof`i'=sum(level=="`i'")}

Stata: Create id by group

When doing your data analysis, sometimes you will encounter the following situation: in your dataset, everyone has an unique id. However, their IDs are long and each participant has multiple record (or the dataset is in a long format).

To visualize your data, you need to create a new ID for each individual regardless of how many records each person has. For example, the first person has three records, and we would like to assign a new ID 1 for the first person, and the second person would be 2.

Though it sounds difficult and tedious, it is not difficult to do so.  

egen id = group(oldid)

Just one line and your problem will be solved.

Reference: http://www.stata.com/support/faqs/data/group.html

Stata tutorial index

This is an idex page for Stata tutorial I have written on this blog. I will update this index if I write more. This index also reminds me what I should have written.

Data management

Stata: How to deal with missing values?

Export

Stata: How to export descriptive statistics tables?

Stata: Output correlation table

Stata: Export OLS regression table to Word or Excel

Stata: Export Logistic Regression (Coefficient/Odds ratio) to Word or Excel

Graph

Stata: Draw regression lines across groups

Stata: Drawing regression lines across groups

When performing regression, one commonly compare groups, and the simple way to observe the differences is through graphs. Stata offers several ways to draw graphs. Here are two options.
 


Option 1:

regress inc educ male, beta
predict incfnoi if male==0
predict incmnoi if male==1
twoway (connected incmnoi educ if male==1, lcolor(black) ///
lpattern(dot) msymbol(diamond) msize(large)) ///
(connected incfno educ if male ==0, lcolor(black) ///
lpattern(solid) msymbol(circle) msize(large)), ///
ytitle(Income in thousands) xtitle(Education) ///
legend(order(1 "Men" 2 "Women")) scheme(s2manual)

This code snippet is from A gentle introduction to Stata. The results are presented above.
Option 2:

Install the two packages first:

net describe postgr3, from(http://www.ats.ucla.edu/stat/stata/ado/analysis)
net install postgr3.pkg
net describe xi3, from(http://www.ats.ucla.edu/stat/stata/ado/analysis)
net install xi3.pkg

followed by the code:

xi3: regress inc educ male, beta
postgr3 educ, by(male) table

Just two lines, isn’t that cool? Umm? Do you notice anything wrong? If you don’t like  footnotes, such as yhat_, male==0, you can click the highlighted part below and make some changes.



Select the area you wish to change, and change it in the area indicated by the red arrow.



This also provides a table with which to ensure everything is correct.



You can get more information on the UCLA website here: http://www.ats.ucla.edu/stat/stata/ado/analysis/postgr3.htm

You can also use predxcat and predxcon, with the command findit to get these two packages. I think that options 1 and 2 are adequate. Option 2 also allows you to draw interaction easily. I’ll talk about this in my next post.

Deals on Stata book

A gentle introduction to Stata 


This is a perfect book for beginners.


There are 13 chapters:
1 Getting started
2 Entering data
3 Preparing data for analysis
4 Working with commands, do-files, and results
5 Descriptive statistics and graphs for one variable
6 Statistics and graphs for two categorical variables
7 Tests for one or two means
8 Bivariate regression and correlation
9 Analysis of variance
10 Multiple regression
11 Logistic regression
12 Measurement, reliability, and validity
13 Appendix: What's next?

I like this book because it covers concepts related to statistics as well as their application in a single book. It also tells you how to interpret the results obtained from the Stata output. For example, on page 178, after running a multiple regression:

regress csat expense percent income high college

the results with multiple regression equation are shown :

predicted csat = 851.56 + .00335 expense – 2.618 percent + .0001 income + 1.63 high + 2.03 college

This book also provides helpful interpretations:

Controlling for four other variables weakens the coefficient on expense from –.0223 to .00335, which is no longer statistically distinguishable from zero. The unexpected negative relationship between expense and csat found in our earlier simple regression evidently is spurious, and explained by other predictors.
Only the coefficient on percent (percentage of high school graduates taking the SAT) attains significance at the .05 level. We could interpret this “fourth-orer partial regression coefficient” (so called because its calculation adjusts for four other predictors) as follows.

This book includes many graphs, and when I learn stats I like to see what the results look like. This helps me to understand and remember the concepts I have studied. Visit A gentle introduction to Stata and find out today's deal on Amazon.

Stata: How to export descriptive statistics tables?

When performing statistics analysis, the first thing you would probably do is to run descriptive statistics. Knowing how to export tables of descriptive statistics can save a considerable amount of time.

If you would like to obtain the exact results that I did, you could use the following code for your  dataset:

use http://twtcsl.org/dataset/gss2000.dta
tab race
tab race sex
sum race sex age income
tab race, gen(d)
rename d1 dwhite
rename d2 dblack
rename d3 dother
tab sex, gen(d)
rename d1 dmale
rename d2 dfemale
sum dwhite dblack dother dmale age income if !mi(age) & !mi(income)

The results:



How do we export it? You can use EDIT-COPY TABLE in stata dropdown menu, or write some code to do the work.

estpost sum dwhite dblack dother dmale age income if !mi(age) & !mi(income)
esttab using sum2.rtf, cells("mean(fmt(2)) sd(fmt(2)) min(fmt(1)) max(fmt(0))") nomtitle nonumber replace

The export table looks like this:


To obtain three digits after the decimal point, change fmt(2) to fmt(3).

If you require more advanced descriptive statistics tables, for example, if you wanted to determine age and income by race, you could use the following codes:

sort race
by race: eststo: estpost sum age income if !mi(age) & !mi(income)
esttab using grp_sum.rtf, cells("mean(fmt(2)) sd(fmt(2))") replace

This result in a neat table:

If you are unable to export tables, check this article

Stata: Export OLS regression table to Word or Excel

and install the estout package.

 

Stata: Dealing with missing values

Dealing with missing values is probably the first thing you do after labeling your variables. Unfortunately, this is not an easy job and many users use inappropriate means to accomplish it. Let’s start at the very beginning.

In Stata 7 and previous versions use only one default missing value “.” (without quote). If you wish to exclude missing values, it would be correct to use the:

if variable !=.

Sample code for OLS regression would resemble the following part:

regress a b if c!=. & d!=.
regress a b c if d!=.
regress a b c d

This would be 100% correct if you used an old Stata dataset; however, if your dataset in had different missing values, this code would be problematic. Stata 8 and later versions allows you do define different types of missing values, each of which begins with a “.” (without quote), such as .a, and .b. Therefore, if you have these missing values in your dataset and you use old code like that above, you would probably obtain inconsistent observation numbers.

The correct way to perform this would be to use if c <. or if  !mi(c). The revised code would similar to:

regress a b if !mi(c) & !mi(d)
regress a b c if !mi(d)
regress a b c d

What if you have 20 variables in your regression? Such if statements often result in very long lines, thereby reducing the readability of your code. There are two easy ways to overcome this: 1) creating a dummy called “touse” with 1 representing valid values for all variables; and 0 for at least one missing value.

gen touse =!mi(y, a, b, c, d)
regress y a b if touse
regress y a b c if touse
regress y a b c d if touse

2) If you don’t like this approach, you can also deal with missing values by using nestreg:

nestreg: reg y (a b) (c) (d)

Stata: Export Logistic Regression (Coefficient/Odds ratio) to Word or Excel

Stata provides two commands for logistic regression: logit and logistic. Logit reports coefficients; whereas logistic reports odds ratios. The general command for logistic regression appears like this:

logit y x
logistic y x

Logit output:


Logistic output:


If you want to export the coefficients to Word or Excel, it is the same as exporting an OLS regression. Here is the code I used:

esttab * using logistic1.csv, b(3) pr2

Exporting odds ratios requires transformation. The key is “eform” at the end of this command:

esttab * using logistic2.rtf, b(3) pr2 eform

and the results in Word look like this:


If you require t-test, you may use the following code:

esttab * using logistic3.csv, cells("b(fmt(3) star)" t(par fmt(2))) pr2 
 

Stata: Outputting correlation tables

There are two ways to export correlation tables from Stata to Word or Excel. The first approach works only on Windows. You first select your correlation table and copy it.






You then paste it into Excel and do some editing.


If you would like stars after your correlation coefficient, run a command like this:

pwcorr X1 X2 X3 X4, star(.05)

and then copy the table as mentioned above. It will look like this when you paste it into Excel



The second way is to use esttab, in which the command looks like this:

estpost correlate x1 x2 x3 x4, matrix listwise
est store c1
esttab * using test_correlation.rtf, unstack not noobs compress

Stata: Export OLS regression table to Word or Excel

Stata is a statistics software package with many neat modules that can help you to reduce your workload. One of my favorite modules is estout, which allows the export of your regression tables directly from Stata to Word documents or Excel. Isn’t that cool?

Website：http://repec.org/bocode/e/estout/installation.html

1. First, install this great module by typing the following command in Stata:
ssc install estout, replace

2. Run one OLS regression (the program can export many regression tables, but for now, we will limit ourselves to one).



3. When you are done, type the following:

esttab using test.rtf

4. You can find this file in my document\stata folder. It appears ike this:

5. If you are using have hierarchical regression/ nested regression, things can become a bit complicated. You have to store it by typing est store m1 after running your first regression. It would be look like this:

regress y x1 x2
est store m1
regress y x1 x2 x3 x4
est store m2
regress y x1 x2 x3 x4 x5 x6
est store m3
esttab * using test.rtf, replace 


6. The export file would appear like this:



7. If you would prefer the output to be in Excel format, you can use test.csv.