Difference between revisions of "Main Page/Research/MSB/Scripts"
From phurvitz
Phil Hurvitz (talk | contribs) (→Scripts to parse the MSB data on the receiving Linux server) |
Phil Hurvitz (talk | contribs) |
||
| Line 1: | Line 1: | ||
__FORCETOC__ | __FORCETOC__ | ||
==Scripts to push and pull MSB data== | ==Scripts to push and pull MSB data== | ||
| − | * Raw MSB data are downloaded from the device onto a local PC. This script pushes data to my storage server | + | * Raw MSB data are downloaded from the device onto a local PC. This script pushes data to my storage server '''[[/msb_push_data.pl|msb_push_data.pl]]'''. |
| − | * Data are sucked from my server to Jonathan's server and then the parse and classify routines are run. Data are retrieved from Jonathan's server using | + | * Data are sucked from my server to Jonathan's server and then the parse and classify routines are run. Data are retrieved from Jonathan's server using '''[[/msb.get.data.R|msb.get.data.R]]'''. |
==Scripts to parse the MSB data on the receiving Linux server== | ==Scripts to parse the MSB data on the receiving Linux server== | ||
| − | * Parsing SMS files, standardizes all SMS into a single format, in a single file, | + | * Parsing SMS files, standardizes all SMS into a single format, in a single file, '''[[/preprocess_sms.R|preprocess_sms.R]]''' |
| − | * Joining MSB and GPS data, calculates distance to previous and next points, makes WASPN83 coordinates: | + | *: This creates a single file form which individual subjects' sms records can easily be extracted. |
| − | * To parse the binary | + | * Joining MSB and GPS data, calculates distance to previous and next points, makes WASPN83 coordinates: '''[[/join_data.R|join_data.R]]''' |
| − | *: This requires pre-processing to CSV via | + | * To parse the binary '''MyExperience.sdf''' file into a CSV ASCII file: '''[[/process_sdf.R|process_sdf.R]]''' |
| + | *: This requires pre-processing to CSV via '''[[Media:MyExperience_Analyzer.exe|MyExperience_Analyzer.exe]]''' or '''[[Media:MyExperience_Analyzer_new.exe|MyExperience_Analyzer_new.exe]]'''. | ||
*: Because the sdf files may be in different formats, use trial-and-error to see which exe file will open the sdf. If one exe fails, use the other. | *: Because the sdf files may be in different formats, use trial-and-error to see which exe file will open the sdf. If one exe fails, use the other. | ||
| − | *: When the MyExperience Analyzer opens, click ''Queries > Get All Responses'', then ''File > Save As'' and save the file as | + | *: When the MyExperience Analyzer opens, click ''Queries > Get All Responses'', then ''File > Save As'' and save the file as '''Get All Responses.csv'''. |
* To conflate SMS and MSB records | * To conflate SMS and MSB records | ||
| − | *: Where both have GPS timestamps (Subject ID >= 11): | + | *: Where both have GPS timestamps (Subject ID >= 11): '''[[/conflate_msb_sms_gps.R|conflate_msb_sms_gps.R]]''' |
| − | *: Where SMS does not have GPS timestamps (Subject ID < 11): | + | *: Where SMS does not have GPS timestamps (Subject ID < 11): '''[[/conflate_msb_sms_nogps.R|conflate_msb_sms_nogps.R]]''' |
Revision as of 22:44, 17 January 2009
Contents
Scripts to push and pull MSB data
- Raw MSB data are downloaded from the device onto a local PC. This script pushes data to my storage server msb_push_data.pl.
- Data are sucked from my server to Jonathan's server and then the parse and classify routines are run. Data are retrieved from Jonathan's server using msb.get.data.R.
Scripts to parse the MSB data on the receiving Linux server
- Parsing SMS files, standardizes all SMS into a single format, in a single file, preprocess_sms.R
- This creates a single file form which individual subjects' sms records can easily be extracted.
- Joining MSB and GPS data, calculates distance to previous and next points, makes WASPN83 coordinates: join_data.R
- To parse the binary MyExperience.sdf file into a CSV ASCII file: process_sdf.R
- This requires pre-processing to CSV via MyExperience_Analyzer.exe or MyExperience_Analyzer_new.exe.
- Because the sdf files may be in different formats, use trial-and-error to see which exe file will open the sdf. If one exe fails, use the other.
- When the MyExperience Analyzer opens, click Queries > Get All Responses, then File > Save As and save the file as Get All Responses.csv.
- To conflate SMS and MSB records
- Where both have GPS timestamps (Subject ID >= 11): conflate_msb_sms_gps.R
- Where SMS does not have GPS timestamps (Subject ID < 11): conflate_msb_sms_nogps.R
