Difference between revisions of "Main Page/Research/MSB/Data processing/how it works"
From phurvitz
< Main Page | Research | MSB | Data processing
Phil Hurvitz (talk | contribs) |
Phil Hurvitz (talk | contribs) |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | + | __FORCETOC__ | |
| − | + | * Each time the MSB starts, a new session is started. Files on the MSB increment with the pattern | |
| + | ** log.uwar | ||
| + | ** log-0001.uwar | ||
| + | ** log-0002.uwar | ||
| + | ** .... | ||
| − | Each part should be downloaded and processed separately. Use the script [[/msb.get.data|msb.get.data]] to download all the parts | + | * Downloaded data are stored in separate subdirectories for each subject, named '''part_00''', '''part_01''', etc., per MSB power-cycle session |
| + | ** Each part should be downloaded and processed separately. | ||
| + | ** Use the script [[../../Scripts/msb.get.data.pl|msb.get.data.pl]] to download all the parts in one step. | ||
| − | Once downloaded, use the R script [[/read.msb.files|read.msb.files]] to generate csv files. | + | * Once downloaded, use the R script [[../../Scripts/read.msb.files.R|read.msb.files.R]] to generate csv files. |
| − | To remove duplicate records | + | * There may be records with duplicate timestamps. To remove duplicate records from the class.csv file, use the perl script [[../../Scripts/msb_remdupes.pl|msb_remdupes.pl]]. |
| − | The csv files can be related by timestamps (''i.e.'', '''phone_log''' and '''class''' by the '''date''' field). | + | * The csv files can be related by timestamps (''i.e.'', '''phone_log''' and '''class''' by the '''date''' field). |
Latest revision as of 17:12, 27 January 2009
- Each time the MSB starts, a new session is started. Files on the MSB increment with the pattern
- log.uwar
- log-0001.uwar
- log-0002.uwar
- ....
- Downloaded data are stored in separate subdirectories for each subject, named part_00, part_01, etc., per MSB power-cycle session
- Each part should be downloaded and processed separately.
- Use the script msb.get.data.pl to download all the parts in one step.
- Once downloaded, use the R script read.msb.files.R to generate csv files.
- There may be records with duplicate timestamps. To remove duplicate records from the class.csv file, use the perl script msb_remdupes.pl.
- The csv files can be related by timestamps (i.e., phone_log and class by the date field).
