• Question: How to Plot Data From One File In Defined Time Slices in One

    From John L. Males@21:1/5 to All on Wed Jul 3 18:07:30 2024
    Hello,

    Any tips, insight, pointers, et al would be most appreciated. I
    have spent weeks of hundreds of ideas (mine, research, or
    combination of) would be most appreciate.

    In essence what I am doing is offsetting each line of time using x:y+TimePerLine*yoffset via a scaling factor of yoffset to plot in
    one graph no multiplot for each hour TimePerLine (3600 seconds) to
    start. Hence the two plot lines I am trying to find way to plot
    with goal of seismic helicorder with gnuplot:

    plot '<cat' using ($1)-TimeSecondsGraphStart :($2)+ (VerticalSensitivity/24*16)+Offset with lines linestyle 9

    plot '<cat' using ($1)-TimeSecondsGraphStart-3600 :($2)+ (VerticalSensitivity/24*12)+Offset with lines linestyle 9 set

    The reason for the "($1)-TimeSecondsGraphStart" is because "set
    xrange [0:TimeSecondsDurationGraphLine]" as the input data
    timestamp in "%S" is actual time of data. I did not pre-process the
    data timestamps of data sent to gnuplot relative to zero. Data is
    mostly less than 0.003, often less than 0.0001.

    I am skipping all but the two lines (two plot statements) of
    gnuplot code in my gnuplot source as these are the two lines at
    heart of my challenges I am working with test in order to achieve a
    helicorder with gnuplot.

    If both lines are as above the one, the first plot will plot as
    expected, and second will say ".... line foo: warning: Skipping
    data file with no valid points".

    Same ".... line foo: warning: Skipping data file with no valid
    points"" occurs if I continue first plot line to second line
    without using "plot" on second line.

    If I comment out one of these plot statements then each respective
    plot statement creates a graph with the correct data at the correct
    vertical displacement. In this case on the 16 or 12 vertical
    displacement. For the moment I am using hard coded 3600 for
    seconds per each vertical line, 24 for hours of graph period, and
    12/16 as respective vertical line displacement position. The code
    already has variables for these specific or calculated from base
    values, but I hardcoded to ensure the time segment calculation is
    correct and to ensure the vertical calculation position is very
    clear for this two line plot test before I generate likely via
    for/do for the variable specified graph duration, time per line,
    vertical displacement.


    Background
    ==========

    I have been using gnuplot for years to plot various csv data code I
    write generates on 24/7/365 basis. I am stumped in how to achieve
    with gnuplot how to create what is known in seismic domain as a
    helicorder. I have tried many many ideas of my own and many
    variations of ideas or variations of those ideas via weeks of
    searching wide variety of sources and search venues.

    Before I attempted this I did the really simple plot of 24 hours of
    data as single line for full 24 hours on graph that is easy to do.
    With that single line graph for purposes of testing I add a fixed
    value to the y and depending on of plus or minus value the graph
    would shift towards top or bottom of graph. I also did tests on
    same single plot by adding +- value to x and graph shifted left or
    right per value with of course no data plotted on side of data
    opposite shift value as would expect.

    I will state upfront many will suggest multiplot, but other than
    one special case of multiplot I will expalin that I tried is not
    appropriate as the data of each multiplot is contained within the
    limits of that multiplot, i.e. the graph data cannot exceed the
    borders unless I have missed in my weeks of searching not to be
    case.

    One link with various examples of helicorders and why as far as I
    can determine multiplot will not allow a gnuplot like helicorder to
    transcend many horizontal (time) segments:

    <https://earthquake.usgs.gov/monitoring/seismograms/examples.php>

    For the moment my concern is how to graph multiple lines such that
    will look like the example helicorders.

    For those that want to have copy of data bear in mind the data for
    a 24 hour helicorder in generally 30-60 million million lines of
    data depending on if 100 or 300 samples a second for three axis of
    Z, N, and E for about 0.5TB data before compressing. The data
    format in simplest form for one channel is in format of "%s.%N,
    %e". Unlike most seismic recording systems that output the ADC
    counts for the input ADC voltage my ADC code outputs voltage. I
    use voltage so if the ADC changes (and will as I work on code for
    better ADCs) the same instrument will record in the data file same
    data such that one does not need to "scale" count data due to
    different +- input sensitivity range ratings, and/or PGA settings
    ADCs have (varies from +-0.6V to +-10V) then layered difference if
    ADC is fully differential, bipolar differential, or single ended.
    There are other variables related to AC and instrument, but noted
    are most common.

    The ADC currently being used is set to +-0.256V as that is most
    sensitive for ADC. ADC code is outputting voltage based on ADC
    sensitivity. The counts from ADC is converted by code for ADC with
    respect to ADC sensitivity.

    What I am trying to do is plot one data file of 24 hours of data
    into 24 lines of one hour plots to start. Once I have this
    accomplished then I will use core variables such as period of time
    in hours (common for 24, and 12, but some use 36) helicorder is to
    cover, time in minutes for each line of helicorder (typically 60,
    30, and 15 are used all quite commonly). One side of helicorder
    will have minute start for data line and maybe date as well. Other
    side of graph may have local time, end time in munutes of line, or
    vertical scaling (such a nm of movement information). For the
    purposes of this question I am only interested in how to plot one
    data file split by uniform time length segment of data across
    multiple lines of time, i.e. ignore how to address vertical
    labelling of vertical part of helicorder.

    To cover the one special case of multiplot some of my research
    results suggested if I placed "set multiplot" before and "set
    nomultiplot" after the set of plot statements that would achieve
    the desired result. Sadly the exact results as stated without the
    related multiplot statements occurred. I cannot recall how many
    ideas I tried that were simply not acceptable to gnuplot at all,
    but tried just in case even if to me made no sense or would not
    work just to be sure.



    John L. Males
    Toronto, Ontario, Canada
    03 July 2024 18:07 +0000 UTC
    03 July 2024 14:07 -0400 EDT


    __________________________________________________________________

    V01.06.00
    2024-07-03 17:11:37.710639223+0000-UTC Time: 1720026697.710639223
    PC/System time

    2024-07-03 17:11:37.819672 (+0000) +0.000006 +/- 0.000054
    localhost ::1 s2 no-leap

    2024-07-03 17:11:38.66783 (+0000) -0.001678 +/- 0.018325
    1.ca.pool.ntp.org 216.197.228.230 s1 no-leap

    Linux 5.10.0-23-amd64 #1 SMP Debian 5.10.179-1 (2023-05-12)

    Debian GNU/Linux 11 (bullseye)
    CustomLiveJLMAdvancedWithuC-MX-21.3_x64 Wildflower
    20230709-0256+0000-UTC cat /proc/cpuinfo (Selected):

    model name : Intel(R) Pentium(R) CPU G4560T @ 2.90GHz

    vmstat -s :

    32757020 K total memory
    13972428 K used memory
    9149956 K active memory
    16965800 K inactive memory
    3408380 K free memory
    4 K buffer memory
    15376208 K swap cache
    0 K total swap
    0 K used swap
    0 K free swap
    10205965 non-nice user cpu ticks
    120469430 nice user cpu ticks
    41361092 system cpu ticks
    118919900 idle cpu ticks
    36736 IO-wait cpu ticks
    0 IRQ cpu ticks
    814678 softirq cpu ticks
    0 stolen cpu ticks
    27256401 pages paged in
    3998951 pages paged out
    0 pages swapped in
    0 pages swapped out
    3539710811 interrupts
    1966169522 CPU context switches
    1717501612 boot time
    798709 forks

    /proc/vmstat (Selected) :

    pgpgin 27256401
    pgpgout 3998951
    pswpin 0
    pswpout 0
    pgfree 11380358641
    pgfault 6146587069
    pgmajfault 92199

    /proc/meminfo (Selected) :

    Mlocked: 17368 kB
    VmallocTotal: 34359738367 kB
    VmallocChunk: 0 kB
    HugePages_Total: 0

    ps -A (Selected) :

    %CPU START TIME C CLS COMMAND TIME NI PID POL
    PRI SZ RSS VSZ SIZE MAJFL MINFL SCH STAT TIME WCHAN

    3.4 Jun 4 1453:05 3 TS kswapd0 1-00:13:05 0 70 TS
    19 0 0 0 0 0 0 0 S 1-00:13:05 -

    __________________________________________________________________

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEELO+C3LyeAem0WQtdprsqY7y+VzUFAmaFk2IACgkQprsqY7y+ VzWp6xAAlcZ/YX7pEkdItzbItzeT+5Mo3/gnF5r6JCaW02kzM/6Hmhz5yxX11Qep MFnzP2/4jNKHm2phme9/z1zAuDkVqL41G12r9XK2NX45mW3fpy+GD+X0xJr0xWyV +ZdAlGkftjW3mnxqkrxaIXttV+dY5BanL7bfjGJk/A0C+LMZ9y+mFDBSh5pmp2Vv uXNdwxYQph8vnY98jUy3y6khE45anuQUBtPPRtL2w2DgL5FId8h0Tn+1YCJltL4R SIS4dxzpMRkSZLCwNoMBR1ookaMeH5tzLGjajocYZPAF8kkmwUBYuqcmUEt+baHg MJX9I/+usD/q9NIRmNV7yHCN7G5LwElPmWavdeGi34wDvIgSILpNn2r5wqQRZwd7 7KOQbDF9Hju3Jy1cJeB9QC+uy4EUrIPFcG6RZop6O30I668lrBAAI4joLhHaB+/l 5aoOZqUlA5TUm6j58kWbnQat9B/be2m4xdk5ZSnfzWMMRmnrj1geRldOuVDPCzqV wxf5Y+V4iRBrckf9hF+7u36P04ep6zU4OqfP4oqu1EUUalw8KXh4LTilg9c7qUok W8AGQIuFoTtt3NdxCJSw0bNGO4yzIt/AF8340472SXiV4MQmzKxk3AjytcYMEAS/ zX3KGKlmYlnClS4/9SGRYMDw4aI6wo4hijRy3WU/v3ylMi9Ei/A=
    =clda
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)