User Tools

Site Tools

Trace:
vd

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
vd [2021/08/06 13:02]
dblume Added PG&E example 		vd [2023/04/10 10:27] (current)
dblume
Line 25: Line 25:
 | ; | Extract regex to new column. Ex, ''(video|audio)'', ''(^..)'' or ''(^([STUVW]%%...%%|..))'' | | ; | Extract regex to new column. Ex, ''(video|audio)'', ''(^..)'' or ''(^([STUVW]%%...%%|..))'' |
 | %%^%% | rename the column. Might have to be "product_id" or "platform_id" | | %%^%% | rename the column. Might have to be "product_id" or "platform_id" |
 +| = | Use Python function to create new column. Ex, hex to dec: ''int(curcol,16)'' |
 +| : | Split column by regex |
 | - | Hide column | | - | Hide column |
 | S | Go to "Sheets" sheet, to select another sheet to format. | | S | Go to "Sheets" sheet, to select another sheet to format. |
Line 34: Line 36:
 | " | Open duplicate sheet with only selected rows | | " | Open duplicate sheet with only selected rows |
  
-==== Case Study: Exported CSV from PG&E ====+===== Inspecting Columnar Data =====
  
-PG&E CSVs come with 5 rows of metadata followed by Type, Date, Start Time, End Time, Usage, Units, Cost, Notes columns. Delete the five rows of metadata in a text editor, or use ''tail'' to remove them like so:+^ Key ^ Meaning ^ 
 +| I | Describe all columns, errors, distinct, mode, mean, median, stdev, etc. | 
 +| i | Add a column of incrementing numbers (useful for '.' charts) | 
 +| . | Requires an "important" numeric column for row data. Make chart. | 
 +| O | Options to enable "numeric_binning" and set number of "histogram_bins" (use 'e') | 
 +| F | Frequency table of row counts, or histogram if numeric_binning is true |
  
-    tail +6 pge_electric_interval_data.csv | vd -f csv -+====== Case Study Link: Exported CSV from PG&E ======
  
-Then prepare your PG&E data like so:+Visit [[vd-pge]].
  
-^ Key ^ Meaning ^ +====== Cast Study: Merging Two Tables, logs and metadata ======
-| - | Hide columns TYPE, END TIME, UNITS and NOTES | +
-| C | Go to column mode and... | +
-| t | Select the DATE and START TIME columns | +
-| & | Make a new column that merges them | +
-| q, - | Quit the Column mode, hide DATE and START TIME columns | +
-| O | Go to options mode and... | +
-| e | Set ''disp_date_fmt'' to ''%Y-%m-%d %H:%M'' | +
-| q | Quit options mode. | +
-| @!, %, $ | Set DATE_START_TIME to date format and important, USAGE to float, COST to currency | +
-| = | Add a column, enter ''COST/USAGE'', (Make it float with ''%'') | +
-| %%^%% | Rename COST/USAGE to ''kWh cost'' | +
-| . | Select columns to graph them. Notice rate changes. Notice times of high use. |+
  
 ==== Protip: Use column view to set multiple columns at once ==== ==== Protip: Use column view to set multiple columns at once ====
vd.1628280146.txt.gz · Last modified: 2021/08/06 13:02 by dblume

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki