Iot: Error while trying to initialize pin interrupts.

[chrishamm]

I am using the GpioController on a Raspberry Pi 3B+ with an interrupt routine. The code is relatively simple, I initialize the GpioController on program start and register an interrupt for a single pin for a rising event:

_pinController = new GpioController(PinNumberingScheme.Logical); _pinController.OpenPin(Settings.TransferReadyPin); _pinController.SetPinMode(Settings.TransferReadyPin, PinMode.InputPullDown); _pinController.RegisterCallbackForPinValueChangedEvent(Settings.TransferReadyPin, PinEventTypes.Rising, OnTransferReady); if (_pinController.Read(Settings.TransferReadyPin) == PinValue.High) { _transferReadyEvent.Set(); }
...
private static void OnTransferReady(object sender, PinValueChangedEventArgs pinValueChangedEventArgs) { _transferReadyEvent.Set(); }
After a while my program crashes randomly with this stack trace:
Unhandled Exception: System.IO.IOException: Error while trying to initialize pin interrupts. at System.Device.Gpio.Drivers.SysFsDriver.WasEventDetected(Int32 pollFileDescriptor, Int32 valueFileDescriptor, Int32& pinNumber, CancellationToken cancellationToken) at System.Device.Gpio.Drivers.SysFsDriver.DetectEvents() at System.Threading.Thread.ThreadMain_ThreadStart() at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location where exception was thrown --- at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart()
I am using System.Device.Gpio 0.1.0-prerelease.19176.1 and IoT.Device.Bindings 0.1.0-prerelease.19176.1 on a recent Raspbian installation. Any idea why this problem occurs?

[joperezr]

Hey @chrishamm thanks for reporting this. We actually started seeing this as well in some of our tests so we are currently investigating the issue. Once we have a fix, I'll update this issue with a package version of System.Device.Gpio that doesn't have the problem any longer.

[joperezr]

Hey @chrishamm we have made some fixes into the callback registration and have validated that most of the tests we have for eventing scenarios are succeeding, do you mind giving this another try with our new packages? The new package version for System.Device.Gpio is 0.1.0-prerelease.19273.3 and it is located in our blob feed which you can easily add a reference to the blob feed by adding a NuGet.config file next to your .csproj that has the following contents:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="dotnet-iot" value="https://dotnetfeed.blob.core.windows.net/dotnet-iot/index.json" />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
  </packageSources>
</configuration>

[joperezr]

I'm closing this as I can't repro the issue any longer, but please feel free to reopen if you still see it after consuming the latest package that I mentioned above.

[chrishamm]

I haven't got any more IOExceptions either, so thanks for notification and for the fix!

Yet I am facing a new problem: When I register a pin change callback, it is not called every time when an event occurs. I need to react to such an event more than 1000 times per second and the GpioController seems to max out at approx. 230 pulses/sec, although direct polling via Read() tells me it the level changes far more than 400 times per second. This problem is not present in WiringPi / Raspberry# but perhaps that library uses a different way to access the RasPi's GPIO chip.

[joperezr]

Yes, we currently don't have a good way of checking the throughput for a high concentration of events. The way we do it today is by using epoll and the sysfs Apis so that we get a notification everytime the value changes. That said, epoll solution doesn't scale very easily and the value file doesn't always detect the change if it happens too fast. AFAIK WiringPi was doing something similar, but we will have to check again in order to get that analyzed better. I would suggest for you to log a new issue against us saying that the throughput of events and callbacks that get fired isn't good for your needs, with a bit of more explanation into your actual scenario so that we can try to see how to optimize better.

[chrishamm]

Thank you, will do!

[chrishamm]

Hmm, I thought I'd give System.Device.Gpio a try once again but unfortunately the same problem popped up again. I am using these versions now:

<PackageReference Include="Iot.Device.Bindings" Version="0.1.0-prerelease.19375.5" /> <PackageReference Include="System.Device.Gpio" Version="0.1.0-prerelease.19375.5" />

And two callbacks for Falling and Rising. The error is the same as last time:

System.IO.IOException: Error while trying to initialize pin interrupts. at System.Device.Gpio.Drivers.SysFsDriver.WasEventDetected(Int32 pollFileDescriptor, Int32 valueFileDescriptor, Int32& pinNumber, CancellationToken cancellationToken) at System.Device.Gpio.Drivers.SysFsDriver.DetectEvents() at System.Threading.Thread.ThreadMain_ThreadStart() at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location where exception was thrown --- at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart()

Hence I would reopen this issue but it looks like I am not allowed to do that.

[joperezr]

@chrishamm can you please provide a simple repro project so that I can take a look? If you could also provide details on where are you trying to run it (like which board are you running in and what OS is installed) that would be great so we can investigate. Also as a side note, we just release preview 7 this morning into NuGet, so you might want to give those new packages a try just in case.

[krwq]

@chrishamm for frequent events I'd recommend to just use a simple loop for now. Events are currently not very well suited for stuff happening frequently (especially SysFs implementation) - I wouldn't recommend using them for anything other than simple scenarios like button.

[chrishamm]

@joperezr Have a look at https://github.com/chrishamm/DuetSoftwareFramework/blob/master/src/DuetControlServer/SPI/DataTransfer.cs. This program establishes an SPI connection to a Duet 3 (a 3D printer board in development/beta testing) and whenever the Duet 3 is ready to exchange data, a GPIO pin is toggled. This happens several times per second.

Because of the exception from the GpioController I copied most of your UNIX SPI implementation to my own library and implemented my own event-based input GPIO class that works via the Linux GPIO chip device. I licensed this particular class under MIT license so you can adopt it if you like. See https://github.com/chrishamm/DuetSoftwareFramework/blob/master/src/LinuxDevices/InputGpioPin.cs and https://github.com/chrishamm/DuetSoftwareFramework/blob/master/src/LinuxDevices/Interop/Libc/Interop.ioctl.cs

[chrishamm]

@krwq A loop is no good solution because it either introduces latency or unnecessarily high CPU usage. Querying the GPIO chip device directly is a much better solution.

[joperezr]

I agree that our current implementation for the SysFS driver is not ideal when polling. I was trying to instead port this to use ioctl instead, but couldn't find good documentation on how to do this using the GPIO chip device. I found a tool in the linux kernel repo that uses this mechanism for polling but that code isn't under MIT license so I was not able to look at the code. Do you mind pointing me to the docs where you based your implementation from so that we can move our eventing to do the same? That way, you could also remove your private implementation from the code and just use our library instead 😄

[chrishamm]

Yes, I hope I can move over to your library again when .NET Core 3 is out :)

Have a look at this crate, its license seems to be fully compatible with the MIT license: https://docs.rs/crate/gpiochip/0.1.1/source/src/lib.rs Essentially, the gpiochip dev node seems to be controllable entirely via ioctl calls so it would be nice if you came up with a new GPIO controller implementation.

Just be aware that pins allocated via SysFS cannot be controlled via the gpiochip dev node (at least not on my RaspPi 3B+).

[joperezr]

We weren't able to fix this in time for 3.0 snap but we will work on this for vNext in order to make our SysFS driver eventing more reliable. We will add this issue to the roadmap for vNext.

[pgrawehr]

@joperezr : I think this ticket can be closed (again). Interrupt handling is now much more reliable, can usually handle 1khz without problems and I've never seen the above exception again.

[joperezr]

Agreed. @chrishamm feel free to re-open if you still hit issues.

[chrishamm]

Well, I doubt 1khz is sufficient for my current application but that isn't related to this ticket. Are you using gpiochip in .NET Core 3 or still the old-fashioned polling approach? I haven't checked the sources yet.

[pgrawehr]

I've tested up to about 7kHz. Above that it gets flacky and more and more triggers seem to get lost. For my project, I've therefore added a counter IC to ensure I'm loosing as few triggers as possible. But I wasn't able to test the full setup yet.

The current implementation uses SysFs interrupt handling (it's not using polling, but the implementation has a significant overhead) or - if available - Libgpiod, which is the new gpio kernel driver. This one is more robust and should generally be faster.

[bcpberry]

I was getting this exception quite consistently (i.e. more than daily) using the current release version (1.0.0) on a test platform I am running. I updated to the current prerelease version (1.1.0-prerelease.20153.1). It seems much better and my system has been running continuously for about a week. However it has just now crashed again with the same exception. As far as I can tell all inputs were stable at the time and I cannot offer any further information about what may have contributed to the crash.

Platform is:
• Raspberry Pi 3A+
• Linux 4.19.97-v7+ #1294 SMP Thu Jan 30 13:15:58 GMT 2020
• .NET Core 3.1.5

[pgrawehr]

I must say, I've also seen this happening, but only after several hours. I haven't had time to investigate it, though. I would assume it is some low-level operating system call that fails once in a while. But since there's not much more information available, the only workaround would probably be a retry.

[bcpberry]

I agree @pgrawehr and thought the same, but the problem is where does one put the retry? This is occurring in a background thread or ISR (from what I can see).

How does one re-open this issue so that it can be put back on the TODO list?

[joperezr]

Given there is no consistent repro for this, the only thing that I would reopen this for is to add more diagnostic info into that exception that might help track down what the underlying issue is, so I'm reopening it for that purpose only. Once we submit a PR with more diagnostic info to the exception, then we can close this issue again with that PR and once people start getting exceptions in their apps they can open new issues with this new diagnostic info.

[tjWren]

.NET Core 3.1.102

PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
Linux 4.19.108-v7+ armv7l

I'm having the same issue after a couple hours I get:
Unhandled exception. System.IO.IOException: Error while trying to initialize pin interrupts.
at System.Device.Gpio.Drivers.SysFsDriver.WasEventDetected(Int32 pollFileDescriptor, Int32 valueFil$
at System.Device.Gpio.Drivers.SysFsDriver.DetectEvents()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback$
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback$
at System.Threading.ThreadHelper.ThreadStart()

and the program crashes.

I have two diffrent systems running and the other is fine. This one is running a SignalR Client

Code:
using System;
using System.Device.Gpio;
using System.Threading;

namespace device.Classes
{
public class HWInput : IDisposable
{

    public event EventHandler IOChanged;
    public int GPIONumber { get; set; }
    public GpioController GPIO { get; set; }
    public bool Value { get; set; }
    public string Name { get; set; }
    public HWInput(int gpioNum, string name,GpioController gpio)
    {
        GPIO = gpio;
        GPIONumber = gpioNum;
        Name = name;
        if (!GPIO.IsPinOpen(GPIONumber))
        {
            GPIO.OpenPin(GPIONumber, PinMode.InputPullUp);
        }
        else
        {
            GPIO.SetPinMode(GPIONumber, PinMode.InputPullUp);
        }
        GPIO.RegisterCallbackForPinValueChangedEvent(GPIONumber, PinEventTypes.Rising | PinEventTypes.Falling, UpdateValues);
        UpdateValues(this, null);
    }

    private void UpdateValues(object sender, PinValueChangedEventArgs pinValueChangedEventArgs)
    {
        Thread.Sleep(150);
        Value = GPIO.Read(GPIONumber) == PinValue.High;
        IOChanged?.Invoke(this, new EventArgs());
    }

    public void Dispose()
    {
        GPIO.UnregisterCallbackForPinValueChangedEvent(GPIONumber, UpdateValues);
        GPIO.ClosePin(GPIONumber);
    }
}

}

Class is called
GPIO = new GpioController();
Input1 = new HWInput(20, "Digital Input 1", GPIO);//GPIO20

This is happening very inconsistently but I have 14 Raspberry Pis and 10 will crash after a couple hours.

[pgrawehr]

I've recently seen this happening while I was debugging - so it might be a timing issue of some kind.

[tjWren]

The only difference between my two devices is this one is calling dbus commands for omxplayer and is a SignalR client. I don't think I saw this happening until I added dbus controls.