Openbci_gui: ArrayOutOfBoundsException when streaming 16 channels over network widget

[Joe-Westra]

Problem

When trying to stream data out over the network using the network widget an ArrayOutOfBoundsException is thrown if all of the following conditions hold:
-The number of channels is 16 (either from playback file or from live data)
-The protocol in the network widget is "UDP", "OSC", or "Serial"
-The Data Type is "Time Series"
-"Filters" are turned on

Expected

As the unfiltered data stream works fine if "Filters" are set to the off position, I would expect that filtered data would stream out once the Filters are set to On.

Operating System and Version

Linux Ubuntu 18.04.3

GUI Version

4.1.6

Running standalone app

Running from Processing

Type of OpenBCI Board

Cyton+Daisy
The same issue is experienced when running via the "playback" function with any recorded data.
NOTE: a 16 channel sample included in the /SampleData would be a nice addition!

Are you using a WiFi Shield?

Yes

Console Log

I modified the sendTimeSeriesData() method within DataProcessing to print out some extra information to aid in debugging purposes:

// TIME SERIES FILTERED
        } else if (filter==1) {
            if (this.protocol.equals("OSC")) {
                for (int i=0;i<nPointsPerUpdate;i++) {
                    msg.clearArguments();
                    for (int j=0;j<numChan;j++) {
                        msg.add(dataBuffY_filtY_uV[j][start+i]);
                    }
                    try {
                        this.osc.send(msg,this.netaddress);
                    } catch (Exception e) {
                        println(e.getMessage());
                    }
                }
            } else if (this.protocol.equals("UDP")) {
                //Added: provide the length of the array
                println("*length of the array: " + dataBuffY_filtY_uV[0].length);

                for (int i=0;i<nPointsPerUpdate;i++) {
                    if (packetNum == MAXPNUM){
                      packetNum = 0;
                    } else {
                      packetNum += 1; 
                    }
                    //Added: packetNumber included in each sample sent
                    String outputter = "{\"type\":\"eeg\",\"packetNumber\":\""+ packetNum +"\",\"data\":[";

                    for (int j = 0; j < numChan; j++) {
                      //Added: Print the current buffer contents
                      println("*buffer data: " + str(dataBuffY_filtY_uV[j][start+i]) + "\tj: " + j + "\tstart+i: " + str(int(start) + int(i)) );
                        outputter += str(dataBuffY_filtY_uV[j][start+i]);
                        if (j != numChan - 1) {
                            outputter += ",";
                        } else {
                            outputter += "]}\r\n";
                        }
                    }
                    try {
                        this.udp.send(outputter, this.ip, this.port);
                    } catch (Exception e) {
                        println(e.getMessage());
                    }
                }
            }
    ....

With the debugging.isVerbose boolean set to true and a the playback from file option used:

Console Log Started at Local Time: 2019-11-07_17-38-33
Screen Resolution: 1920 X 1080
Welcome to the Processing-based OpenBCI GUI!
For more information, please visit: https://openbci.github.io/Documentation/docs/06Software/01-OpenBCISoftware/GUIDocs
Killed:  6104 pts/0    00:00:00 OpenBCIHub
Killed:  6107 pts/0    00:00:00 OpenBCIHub
Launching application from local data dir
OpenBCI_GUI: hubStart: OS Detected: Linux
22.032001
36.924004
40.188
43.554
46.920002
50.388
53.754
57.222
60.588
63.954002
67.217995
70.584
74.052
77.41801
80.784
84.15
87.516
90.882
94.145996
97.614006
100.979996
104.346
107.712
110.976
114.444
117.708
121.176
124.542
127.908005
131.274
134.64
138.006
141.37201
144.84001
148.206
151.57199
154.83601
158.304
161.772
165.138
168.402
171.768
175.134
178.602
182.06999
185.43599
188.904
192.27
195.636
199.002
202.36801
205.73401
208.998
212.46599
215.934
219.198
222.564
226.032
229.5
Local Version: 416, Latest Version: 416
GUI is up to date!
232.86601
236.232
239.59799
242.86201
246.33
249.594
252.95999
256.938
259.794
263.67
267.138
270.504
273.768
277.23602
280.5
OpenBCI_GUI::Setup: Setup is complete!
OpenBCI_GUI::Setup: Is RX mulitcast: false
OpenBCI_GUI::Setup: Has RX joined multicast: false
OpenBCI_GUI::Setup: Sample Data exists in Documents folder.
ControlP5 2.2.6 infos, comments, questions at http://www.sojamo.de/libraries/controlP5
CP: Playback History Size = 2
293.454
Instantiating hub object...
hub IP:127.0.0.1
hub port:10996
Socket[addr=/127.0.0.1,port=10996,localport=49282]
Hub: CheckHubInit: The GUI is connected to the Hub!
sourceList: click! 
setInterface: curInterface: -1
[DEFAULT]: Select a file for playback
Gtk-Message: 17:38:49.016: Failed to load module "canberra-gtk-module"
DataLogging: playbackSelected: User selected /home/abv/Documents/OpenBCI_GUI/Recordings/OpenBCISession_useThisAsATest/OpenBCI-RAW-2019-11-07_10-51-56.txt
OpenBCI_GUI: initSystem: loading playback data from /home/abv/Documents/OpenBCI_GUI/Recordings/OpenBCISession_useThisAsATest/OpenBCI-RAW-2019-11-07_10-51-56.txt
readCSV: %OpenBCI Raw EEG Data
readCSV: %Number of channels = 16
Sample rate set to 1000
readCSV: %Sample Rate = 1000.0 Hz
readCSV: %First Column = SampleIndex
readCSV: %Last Column = Timestamp 
readCSV: %Other Columns = EEG data in microvolts followed by Accel Data (in G) interleaved with Aux Data
OpenBCI_GUI: initSystem: loading complete.  38090 rows of data, which is 38 seconds of EEG data
channel count set to 16
[SUCCESS]: You have selected "OpenBCI-RAW-2019-11-07_10-51-56.txt" for playback. 16 channels found.
Found user playback history file!
initButtonPressed: Calling initSystem()


=================================================
||             INITIALIZING SYSTEM             ||
=================================================

OpenBCI_GUI: initSystem: -- Init 0 -- 24020
OpenBCI_GUI: initSystem: Preparing data variables...
OpenBCI_GUI: initSystem: loading playback data from /home/abv/Documents/OpenBCI_GUI/Recordings/OpenBCISession_useThisAsATest/OpenBCI-RAW-2019-11-07_10-51-56.txt
readCSV: %OpenBCI Raw EEG Data
readCSV: %Number of channels = 16
Sample rate set to 1000
readCSV: %Sample Rate = 1000.0 Hz
readCSV: %First Column = SampleIndex
readCSV: %Last Column = Timestamp 
readCSV: %Other Columns = EEG data in microvolts followed by Accel Data (in G) interleaved with Aux Data
OpenBCI_GUI: initSystem: loading complete.  38090 rows of data, which is 38 seconds of EEG data
OpenBCI_GUI: initSystem: Initializing core data objects
OpenBCI_GUI: initSystem: -- Init 1 -- 24471
OpenBCI_GUI: initSystem: Initializing FFT data objects
OpenBCI_GUI: initSystem: -- Init 2 -- 24495
OpenBCI_GUI: initSystem: Closing ControlPanel...
OpenBCI_GUI: initSystem: Initializing comms with hub....
OpenBCI_GUI: initSystem: -- Init 3 -- 24496
null
"Arial" is not available, so another font will be used. Use PFont.list() to show available fonts.
ChannelController: createChannelSettingButtons: creating channel setting buttons...
null
null
null
null
comPorts = [/dev/ttyS0, /dev/ttyS1, /dev/ttyS2, /dev/ttyS3, /dev/ttyS4, /dev/ttyS5, /dev/ttyS6, /dev/ttyS7, /dev/ttyS8, /dev/ttyS9, /dev/ttyS10, /dev/ttyS11, /dev/ttyS12, /dev/ttyS13, /dev/ttyS14, /dev/ttyS15, /dev/ttyS16, /dev/ttyS17, /dev/ttyS18, /dev/ttyS19, /dev/ttyS20, /dev/ttyS21, /dev/ttyS22, /dev/ttyS23, /dev/ttyS24, /dev/ttyS25, /dev/ttyS26, /dev/ttyS27, /dev/ttyS28, /dev/ttyS29, /dev/ttyS30, /dev/ttyS31, /dev/ttyUSB0]
null
null
null
null
null
null
WM: Powering 4 widgets up, and remapping.
WM: Activating widget [0]
WM: Activating widget [1]
WM: Activating widget [2]
WM: Activating widget [3]
OpenBCI_GUI: initSystem: -- Init 4 -- 25847
in softwaresettings.init()
InitSettings: Saving Default Settings to file!
SoftwareSettings: 4 active widgets saved!
loadGUISettings: Data source loaded: 2. Current data source: 2
WM: Remapping widgets.
LoadGUISettings: Layout 1 Loaded!
Deactivating widget [0]
Deactivating widget [1]
Deactivating widget [2]
Deactivating widget [3]
LoadGUISettings: Applied Widget 0 to Container 1
LoadGUISettings: Applied Widget 2 to Container 7
LoadGUISettings: Applied Widget 3 to Container 9
Networking Data Types || show num dropdowns = 6
LoadGUISettings: Applied Widget 4 to Container 3
LoadGUISettings: Expert Mode Off
Settings: SSVEP Active Channels: [7, 8]
Settings: SSVEP Active Channels: [7, 8]
Networking: Protocol mode set to LSL
Networking Data Types || show num dropdowns = 6
Apply LSL Networking Mode
OpenBCI_GUI: initSystem: -- Init 5 -- Settings Loaded! 26997
[SUCCESS]: Settings Loaded!
TopNav: Updated Settings Button Position
OpenBCI_GUI: getPlaybackDataFromTable: End of playback data file.  Starting over...
number of indexes 38091
Finished filling hashmap
^^^GUI update process file has occurred
Networking: Protocol mode set to UDP
Networking Data Types || show num dropdowns = 6
openBCI_GUI: startButton was pressed...starting data transfer...
startRunning...
[DEFAULT]: Data stream started.
Stream update numChan to 16
nPointsPerUpdate 40
dataToSend len: 640
[DEFAULT]: Network Stream Started
Networking: StringList size=4 [ "TimeSeries", "127.0.0.1", "12345", "1" ]
[DEFAULT]: UDP successfully connected
*length of the array: 23000
*buffer data: -60.862133    j: 0    start+i: 22989
*buffer data: -7.098062 j: 1    start+i: 22989
*buffer data: -43.565296    j: 2    start+i: 22989
*buffer data: -91.99674 j: 3    start+i: 22989
*buffer data: -42.942505    j: 4    start+i: 22989
*buffer data: -8482.759 j: 5    start+i: 22989
*buffer data: -1672.5138    j: 6    start+i: 22989
*buffer data: -118.82869    j: 7    start+i: 22989
*buffer data: -57.493042    j: 8    start+i: 22989
*buffer data: -134.09682    j: 9    start+i: 22989
*buffer data: -63.52128 j: 10   start+i: 22989
*buffer data: -106.881935   j: 11   start+i: 22989
*buffer data: -56.69043 j: 12   start+i: 22989
*buffer data: -165.61578    j: 13   start+i: 22989
*buffer data: -27.020992    j: 14   start+i: 22989
*buffer data: -89.099525    j: 15   start+i: 22989
*buffer data: -152937.0 j: 0    start+i: 22990
*buffer data: -155891.12    j: 1    start+i: 22990
*buffer data: -152688.25    j: 2    start+i: 22990
*buffer data: -66506.32 j: 3    start+i: 22990
*buffer data: -131253.7 j: 4    start+i: 22990
*buffer data: 32433.744 j: 5    start+i: 22990
*buffer data: -144048.4 j: 6    start+i: 22990
*buffer data: -135602.38    j: 7    start+i: 22990
*buffer data: -168274.75    j: 8    start+i: 22990
*buffer data: -136861.58    j: 9    start+i: 22990
*buffer data: -172834.86    j: 10   start+i: 22990
*buffer data: -114720.33    j: 11   start+i: 22990
*buffer data: -155519.12    j: 12   start+i: 22990
*buffer data: -47727.723    j: 13   start+i: 22990
*buffer data: -150168.14    j: 14   start+i: 22990
*buffer data: -131715.1 j: 15   start+i: 22990
*buffer data: 25.916111 j: 0    start+i: 22991
*buffer data: -155942.4 j: 1    start+i: 22991
*buffer data: -152727.97    j: 2    start+i: 22991
*buffer data: -66567.94 j: 3    start+i: 22991
*buffer data: -131293.78    j: 4    start+i: 22991
*buffer data: 32435.354 j: 5    start+i: 22991
*buffer data: -145522.48    j: 6    start+i: 22991
*buffer data: -135660.25    j: 7    start+i: 22991
*buffer data: -168327.2 j: 8    start+i: 22991
*buffer data: -136955.53    j: 9    start+i: 22991
*buffer data: -172891.9 j: 10   start+i: 22991
*buffer data: -114799.45    j: 11   start+i: 22991
*buffer data: -155568.81    j: 12   start+i: 22991
*buffer data: -47789.21 j: 13   start+i: 22991
*buffer data: -150192.39    j: 14   start+i: 22991
*buffer data: -131761.05    j: 15   start+i: 22991
*buffer data: 25.545984 j: 0    start+i: 22992
*buffer data: 2.7095351 j: 1    start+i: 22992
*buffer data: 18.068832 j: 2    start+i: 22992
*buffer data: 40.41126  j: 3    start+i: 22992
*buffer data: -131369.42    j: 4    start+i: 22992
*buffer data: 27962.455 j: 5    start+i: 22992
*buffer data: -148568.38    j: 6    start+i: 22992
*buffer data: -135841.97    j: 7    start+i: 22992
*buffer data: -168428.23    j: 8    start+i: 22992
*buffer data: -137179.72    j: 9    start+i: 22992
*buffer data: -173003.28    j: 10   start+i: 22992
*buffer data: -114981.62    j: 11   start+i: 22992
*buffer data: -155666.42    j: 12   start+i: 22992
*buffer data: -48032.445    j: 13   start+i: 22992
*buffer data: -150237.3 j: 14   start+i: 22992
*buffer data: -131896.97    j: 15   start+i: 22992
*buffer data: 21.78378  j: 0    start+i: 22993
*buffer data: 2.31119   j: 1    start+i: 22993
*buffer data: 15.580187 j: 2    start+i: 22993
*buffer data: 36.37224  j: 3    start+i: 22993
*buffer data: 14.118802 j: 4    start+i: 22993
*buffer data: 1437.1184 j: 5    start+i: 22993
*buffer data: -152908.53    j: 6    start+i: 22993
*buffer data: -136130.23    j: 7    start+i: 22993
*buffer data: -168567.98    j: 8    start+i: 22993
*buffer data: -137507.22    j: 9    start+i: 22993
*buffer data: -173157.58    j: 10   start+i: 22993
*buffer data: -115245.68    j: 11   start+i: 22993
*buffer data: -155804.55    j: 12   start+i: 22993
*buffer data: -48430.887    j: 13   start+i: 22993
*buffer data: -150302.02    j: 14   start+i: 22993
*buffer data: -132111.02    j: 15   start+i: 22993
*buffer data: 15.034733 j: 0    start+i: 22994
*buffer data: 1.606273  j: 1    start+i: 22994
*buffer data: 10.87744  j: 2    start+i: 22994
*buffer data: 27.1636   j: 3    start+i: 22994
*buffer data: 9.532091  j: 4    start+i: 22994
*buffer data: 1612.5603 j: 5    start+i: 22994
*buffer data: 419.98126 j: 6    start+i: 22994
*buffer data: -136486.78    j: 7    start+i: 22994
*buffer data: -168735.92    j: 8    start+i: 22994
*buffer data: -137901.62    j: 9    start+i: 22994
*buffer data: -173342.25    j: 10   start+i: 22994
*buffer data: -115561.78    j: 11   start+i: 22994
*buffer data: -155971.47    j: 12   start+i: 22994
*buffer data: -48929.26 j: 13   start+i: 22994
*buffer data: -150382.0 j: 14   start+i: 22994
*buffer data: -132376.89    j: 15   start+i: 22994
*buffer data: 5.9051895 j: 0    start+i: 22995
*buffer data: 0.5943748 j: 1    start+i: 22995
*buffer data: 4.3471284 j: 2    start+i: 22995
*buffer data: 13.542316 j: 3    start+i: 22995
*buffer data: 3.1149487 j: 4    start+i: 22995
*buffer data: 644.90906 j: 5    start+i: 22995
*buffer data: 178.51337 j: 6    start+i: 22995
*buffer data: -136595.73    j: 7    start+i: 22995
*buffer data: -168884.4 j: 8    start+i: 22995
*buffer data: -138259.3 j: 9    start+i: 22995
*buffer data: -173505.52    j: 10   start+i: 22995
*buffer data: -115841.195   j: 11   start+i: 22995
*buffer data: -156119.73    j: 12   start+i: 22995
*buffer data: -49404.773    j: 13   start+i: 22995
*buffer data: -150451.86    j: 14   start+i: 22995
*buffer data: -132625.61    j: 15   start+i: 22995
*buffer data: -4.2984056    j: 0    start+i: 22996
*buffer data: -0.6319766    j: 1    start+i: 22996
*buffer data: -3.0261204    j: 2    start+i: 22996
*buffer data: -2.1139228    j: 3    start+i: 22996
*buffer data: -4.294787 j: 4    start+i: 22996
*buffer data: -927.54663    j: 5    start+i: 22996
*buffer data: -107.752014   j: 6    start+i: 22996
*buffer data: 2.2439828 j: 7    start+i: 22996
*buffer data: -168998.77    j: 8    start+i: 22996
*buffer data: -138554.9 j: 9    start+i: 22996
*buffer data: -173637.05    j: 10   start+i: 22996
*buffer data: -116070.016   j: 11   start+i: 22996
*buffer data: -156229.88    j: 12   start+i: 22996
*buffer data: -49816.65 j: 13   start+i: 22996
*buffer data: -150506.5 j: 14   start+i: 22996
*buffer data: -132839.55    j: 15   start+i: 22996
*buffer data: -13.77114 j: 0    start+i: 22997
*buffer data: -1.8637608    j: 1    start+i: 22997
*buffer data: -9.878688 j: 2    start+i: 22997
*buffer data: -16.741201    j: 3    start+i: 22997
*buffer data: -11.141836    j: 4    start+i: 22997
*buffer data: -2307.3518    j: 5    start+i: 22997
*buffer data: -374.03052    j: 6    start+i: 22997
*buffer data: -17.426022    j: 7    start+i: 22997
*buffer data: -12.871463    j: 8    start+i: 22997
*buffer data: -138745.3 j: 9    start+i: 22997
*buffer data: -173718.02    j: 10   start+i: 22997
*buffer data: -116220.91    j: 11   start+i: 22997
*buffer data: -156283.06    j: 12   start+i: 22997
*buffer data: -50110.754    j: 13   start+i: 22997
*buffer data: -150538.92    j: 14   start+i: 22997
*buffer data: -132993.28    j: 15   start+i: 22997
*buffer data: -20.863836    j: 0    start+i: 22998
*buffer data: -2.8876584    j: 1    start+i: 22998
*buffer data: -15.076668    j: 2    start+i: 22998
*buffer data: -27.968582    j: 3    start+i: 22998
*buffer data: -15.923041    j: 4    start+i: 22998
*buffer data: -3160.53  j: 5    start+i: 22998
*buffer data: -553.0602 j: 6    start+i: 22998
*buffer data: -33.266754    j: 7    start+i: 22998
*buffer data: -19.7096  j: 8    start+i: 22998
*buffer data: -137857.98    j: 9    start+i: 22998
*buffer data: -173744.16    j: 10   start+i: 22998
*buffer data: -116275.54    j: 11   start+i: 22998
*buffer data: -156291.53    j: 12   start+i: 22998
*buffer data: -50247.637    j: 13   start+i: 22998
*buffer data: -150550.12    j: 14   start+i: 22998
*buffer data: -133065.23    j: 15   start+i: 22998
*buffer data: -24.709085    j: 0    start+i: 22999
*buffer data: -3.5687938    j: 1    start+i: 22999
*buffer data: -18.075706    j: 2    start+i: 22999
*buffer data: -34.69309 j: 3    start+i: 22999
*buffer data: -18.063953    j: 4    start+i: 22999
*buffer data: -3619.7659    j: 5    start+i: 22999
*buffer data: -619.56305    j: 6    start+i: 22999
*buffer data: -43.389004    j: 7    start+i: 22999
*buffer data: -23.336735    j: 8    start+i: 22999
*buffer data: -49.829514    j: 9    start+i: 22999
*buffer data: -173712.16    j: 10   start+i: 22999
*buffer data: -116236.266   j: 11   start+i: 22999
*buffer data: -156261.28    j: 12   start+i: 22999
*buffer data: -50244.55 j: 13   start+i: 22999
*buffer data: -150537.88    j: 14   start+i: 22999
*buffer data: -133053.81    j: 15   start+i: 22999
ArrayIndexOutOfBoundsException: 23000

[Joe-Westra]

As there is no issue when streaming out either filtered or unfiltered data for 8 channels, it _appears to me_ that the issue is due to the start and end values being more or less hard-coded within the Stream class within W_Networking:

// Data buffers
    int start = dataBuffY_filtY_uV[0].length-11;
    int end = dataBuffY_filtY_uV[0].length-1;
    int bufferLen = end-start;
    float[] dataToSend = new float[numChan*bufferLen];

Any feedback on the relevance of 11 and 1 here?

[Joe-Westra]

It appears to me that the start variable should be set to dataBuffY_filtY_uV[0].length - nPointsPerUpdate. Doing so within Stream.updateNumChan allows this to be updated at the correct point:

    private void updateNumChan(int _numChan) {
        numChan = _numChan;
        println("Stream update numChan to " + numChan);
        dataToSend = new float[numChan * nPointsPerUpdate];
        println("nPointsPerUpdate " + nPointsPerUpdate);
        println("dataToSend len: " + numChan * nPointsPerUpdate);

        //Add the code below
        start = dataBuffY_filtY_uV[0].length - nPointsPerUpdate;
    }

[retiutut]

@Joe-Westra Almost thought the above merge was done on the upstream branch for a second. I can test this out today, but I've also gone ahead and made a PR from your development branch.

[retiutut]

Was unable replicate the ArrayOutofBoundsException on MacOS using OSC or UDP with filter On and 16Ch. Let me try again.

[retiutut]

@Joe-Westra Unable to replicate this issue on Ubuntu 18.04.3. :/

[Joe-Westra]

It looks like it might be due to a high sampling rate (1000hz), which is used to calculate nPointsPerUpdate in OpenBCi_GUI.initCoreDataObjects(). What sample rate were you using @retiutut?

[retiutut]

Cyton+Daisy over Dongle is 125Hz. I think Synthetic mode w/ 16Ch is 250Hz. Oh man, I see WiFi shield = yes. ugh.

Thankfully I have an extra WiFi shield to test with. I'll try to replicate again with Cyton+Daisy+WiFi.

[retiutut]

@Joe-Westra Replicated! 🔥 🔥 🔥

Stacktrace points to sendTimeSeriesData() just as you described.

[retiutut]

Fix confirmed using Streaming via OSC to Max/MSP 16ch @ 1000Hz. 🛠