Irremoteesp8266: Sending Raw as hex, possible timing issues.

[greenmikey]

Version/revison of the library used

2.4.0

Expected behavior

I am looking to 'hijack' the signal of an Olympia Infoglobe. This is a Persistence of Vision Caller ID unit. It uses IR to get information from the base to the spinning upper arm. Each bit of data is represented by 1ms pulse @ 38 kHz for a zero and off for 1ms for a one. I am not using all of the functions of the library as I want to send bytes and not durations as I would like to extend this and keep it more flexible for data input.

Basic steps:

1. Initialize signal at 38 kHz
2. Setup data as 8-bit ASCII hex
3. Send header byte (generally 0x00 - 0x08)
4. Send message bytes

Nearly identical code using the Arduino IRremote library works on Arduino Uno.

Actual behavior

If I replace the enabling/disabling code with Serial.print()s I get the correct bit output. This just leaves the modulation or timing as the culprit. When looking at the results in Analysir the timings are inconsistent and do not match my intended 1ms pulse at all. I do not have an oscilloscope so I cannot say how close the modulation is to 38 kHz. I guess it could be a wiring issue but it is driven by a constant current circuit that does work fine on the Arduino version.

Output of raw data from IRrecvDumpV2.ino (if applicable)

This is from Anaysir but should be similar

Raw (25): 3132,-464,1040,-1384,376,-284,980,-4368,344,-1708,1500,-3680,1400,-1340,708,-1428,568,-3376,1624,-2288,716,-3304,700,-1392,1600

Raw (25): 4680,-1584,1384,-4364,676,-1336,312,-96,1152,-3412,1656,-1344,688,-1408,596,-3284,1680,-2304,732,-3284,680,-1312,356,-568,760

Raw (23): 3312,-172,1180,-1300,1656,-4432,564,-1376,1560,-3444,1656,-1368,568,-1428,612,-3420,1544,-2480,552,-3348,372,-1628,1684

Raw (21): 4676,-1312,1628,-4404,700,-1272,1716,-3288,1724,-1272,732,-1288,716,-3260,1720,-2288,728,-3268,724,-1248,1756

Raw (23): 2056,-1300,1620,-4448,676,-1276,1664,-3420,1604,-1428,612,-1272,716,-3384,940,-352,240,-2472,584,-3320,712,-1328,1636

Steps to reproduce the behavior

Run my code?

Example code used

setup without includes

#define IR_LED 4  // ESP8266 GPIO pin to use. Recommended: 4 (D2).

IRsend irsend(IR_LED);  // (pin, not inverted, and use modulation) Set the GPIO to be used to sending the message.

String readText;
unsigned int khz = 38; // 38kHz carrier frequency for infoglobe
unsigned char header[] = {0x04}; //0x01: static, 0x04: scrolling
unsigned char message[35] = {"test"};

void setup() {
  irsend.begin();
  Serial.begin(115200, SERIAL_8N1);
  readText = "test"; //current implementation has this match message variable
}

Loop section

void loop() {  
  Serial.print("Sending raw singal with the message: ");
  Serial.println(readText);

  sendHexRaw(header, 1, 38); //header at 38kHz
  sendHexRaw(message, readText.length(), 38);  // Send a raw data capture at 38kHz.

  delay(5000);
}

Raw send for hex bytes instead of durations

void sendHexRaw(unsigned char *sigArray, unsigned int sizeArray, unsigned int khz) {
  /*  HEADER - the byte determines the transition effect
      MESSAGE - 35 characters max - if sending a message each byte is an ASCII character.
      characters not supported: % & + ; @ [ \ ] ^ _ ` { | } ~
  */
  irsend.enableIROut(khz);
  uint32_t sigTime = micros();
  for (unsigned int i = 0; i < sizeArray; i++) { //iterate thru each byte in sigArray
    register uint8_t bitMask =  0x80; //starting value of bitmask fo each Hex byte
    while (bitMask) { //do 8 times for each bit of the 8 bit byte
      if (bitMask & sigArray[i]) { //its a One bit
        digitalWrite(IR_LED, LOW);
        //Serial.print('1');  //for debug
      }
      else { // its a Zero bit
        digitalWrite(IR_LED, HIGH);
        //Serial.print('0'); //for debug
      }
      while (micros() - sigTime < 1000); //wait here until
      sigTime += 1000;
      bitMask = (unsigned char) bitMask >> 1; // shift the mask bit along until it reaches zero & we exit the while loop
    }
    digitalWrite(IR_LED, LOW);
  }
}

Circuit diagram and hardware used (if applicable)

https://i0.wp.com/www.analysir.com/blog/wp-content/uploads/2013/11/IR-LEd-Constant-current-circuit_sch.png

I have followed the steps in the [Troubleshooting Guide]

Not so much is applicable but I've stepped through it

Has this library/code previously worked as expected for you?

Only on IRremote, not on IrRemoteESP8266

[crankyoldgit]

The short version: You are not really even using our library, hence why the signal looks like garbage. :-) In fact, you are hardly using the IRremote library either. There are already mechanisms to send 8-bit data which you can use in our library.

The longer version:
Pre-fact: The Arduino boards and the ESP8266 chips differ in that there is no(*) hardware-PWM signal generation on the ESP8266s. If you want PWM, it has to basically be done in software.

I am assuming your existing code works on normal arduino chips/boards because the calls you are making to the _IRremote_ library is setting the output pin into a PWM mode for you. Our library can't do that as there is no hardware PWM output.

Don't control the IR LED directly like you are doing with your digitalWrite()s. You need to use our mark() & space() routines in order to get the PWM-like signal for the 38kHz freq. modulation that you want. If you use mark & space calls, your code should be fairly portable between the two libraries.
Effectively mark & space are the lowest level routine you should be using when using the IRremote family of libraries. Below that things get different due to hardware etc. Plus, digitalWrite() on typical Arduino boards (non-ESP) is horribly slow, avoid avoid avoid. mark() in the Arduino IRremote library doesn't use it for that reason.

Your existing code (on the ESP) is literally turning on the IR LED on solid (no PWM, no freq modulation) for milliseconds at a time (depending on the chars being sent). Hence the IR receiver module(s) won't understand it as there is no PWM/Frequency modulation. Also, doing so is likely to burn out the IR LED if you are over-driving it.

Try replacing this code:

      if (bitMask & sigArray[i]) { //its a One bit
        digitalWrite(IR_LED, LOW);
        //Serial.print('1');  //for debug
      }
      else { // its a Zero bit
        digitalWrite(IR_LED, HIGH);
        //Serial.print('0'); //for debug
      }
      while (micros() - sigTime < 1000); //wait here until
      sigTime += 1000;

with (untested):

      if (bitMask & sigArray[i]) { //its a One bit
        irsend.space(1000);  // LED off for 1000 usec = 1msec
        //Serial.print('1');  //for debug  (Note: If you enable this line, you will screw up timings)
      }
      else { // its a Zero bit
         irsend.mark(1000);  // LED on for 1000 usec = 1msec with freq. modulation.
        //Serial.print('0'); //for debug  (Note: If you enable this line, you will screw up timings)

      }

I mentioned there is already a way to send 8-bit data, and it is here (https://github.com/markszabo/IRremoteESP8266/blob/master/src/IRsend.cpp#L432):
However, it is specific only to our library (IRremoteESP8266).
Your use case would probably be (again, un-tested):

// Header char
irsend.sendGeneric(
    0, 0,  // No traditional header
    1000, 0,  // A '1' is a mark of 1msec and no space.
    0, 1000,  // A '0' is a space of 1 msec (no mark at all)
    0,  // No footer mark
    0,  // No gap after this "header" char
    header, 1, // A single 8 bit char.
    38,  // kHz
    true,  // Send the MSB first
    0,  // No repeats,
    50); // Default duty cycle of 50% on and 50% off for the LED during a mark()
// Main message. i.e. The text
irsend.sendGeneric(
    0, 0,  // No traditional header
    1000, 0,  // A '1' is a mark of 1msec and no space.
    0, 1000,  // A '0' is a space of 1 msec (no mark at all)
    0,  // No footer mark
    5000 * 1000,  // A gap of 5 seconds (5000msecs) after this messages. eg. your delay(5000);
    sigArray, sizeArray,
    38,  // kHz
    true,  // Send the MSB first
    0,  // No repeats,
    50); // Default duty cycle of 50% on and 50% off for the LED during a mark()

Note: All code I've added here is done just in the comment, it hasn't been compiled/syntax checked/run/tested/etc at all. Just a guide.

If you want more help in getting this protocol officially supported (it looks pretty easy) we will need to see some raw captures from the remote/working message etc.

All of the above is based just on what you've said it is, and my understanding of what you've said it is. There is a lot of room for error. ;-)

(* Technically there is a hack to use the serial uart to generate a "close-ish" PWM signal in hardware. We don't use it because it limits it to a couple of pins on the ESP8266, and means you can't do serial on those pins)

[greenmikey]

Thanks! I knew I was not using this or the IRremote libraries in their intended way. I was mostly using the IRremote library on Arduino as a convenient way to initialize the PWM and was handling most other things on my own. I knew there was no hardware PWM on the ESP8266 but I assumed (incorrectly) that calling the irread.enableIROut() was initializing it in software on the pin and allowing it to run in the background.

I will update my code and run some tests and let you know how things are doing. I am very excited to read about using 8-bit data because I can move to the world of marks and spaces. It seemed inconvenient when dealing with character input having to convert to marks and spaces. I prototyped it all in python but this is my first time working with c/c++ code so I was a bit hesitant.

I can provide you with lots of information about this protocol. It isn't difficult but it is different in structure compared to things that follow normal protocols.

Thanks again!

Edit:

I've implemented the mark() and space() functions. They do allow me to get some of my signal through but it eventually fails on longer messages from timing issues. Luckily it seems like there is some consistency to my issue. I have pasted some read data from Analysir below.

stuff in loop

sendHexRaw(header, 1, 38); //header at 38kHz
  sendHexRaw(message, readText.length(), 38);  // Send a raw data capture at 38kHz.
  irsend.space(1); //I know of no better way to ensure the LED is off

sendHexRaw Function - removed some of the description to save space, see above for more info

void sendHexRaw(unsigned char *sigArray, unsigned int sizeArray, unsigned int khz) {
  irsend.enableIROut(khz);
  for (unsigned int i = 0; i < sizeArray; i++) { //iterate thru each byte in sigArray
    register uint8_t bitMask =  0x80; //starting value of bitmask fo each Hex byte
    while (bitMask) { //do 8 times for each bit of the 8 bit byte
      if (bitMask & sigArray[i]) { //its a One bit
        irsend.space(1000); // LED off for 1000 usec = 1msec
        //Serial.print('1');  //for debug - ruins timing but ensures you use the correct bits
      }
      else { // its a Zero bit
        irsend.mark(1000);
        //Serial.print('0'); //this will screw up timing but is to ensure you are sending the correct bits
      }
      bitMask = (unsigned char) bitMask >> 1; // shift the mask bit along until it reaches zero
    }
  }
}

Raw (125): 5180,-848,3180,-848,1164,-844,1164,-844,3176,-1852,2168,-844,1160,-848,1160,-2860,2164,-1848,1164,-2852,1160,-844,3176,-1848,1164,-844,2172,-844,1160,-1856,1160,-2852,2168,-1848,2168,-2852,2172,-844,6192,-2856,1164,-2852,1160,-1848,1164,-844,2172,-844,1164,-2852,1160,-844,3180,-1848,1164,-844,5192,-844,6196,-1848,4180,-848,2168,-844,6204,-1844,1168,-1844,3180,-1848,1164,-3852,1168,-1844,1168,-2844,2176,-1844,2176,-2844,1176,-1836,2180,-832,1176,-832,1180,-2836,2184,-828,3196,-828,6204,-1836,1176,-1836,1176,-832,1176,-1836,2208,-804,1204,-804,1204,-2812,2212,-1804,1204,-2812,2208,-1808,1204,-1808,4220,-836,1172,-1812,2208,-2812,1200,-1812,2204,-840,1172

Raw (125): 5188,-844,3180,-848,1160,-848,1156,-852,3172,-1852,2164,-848,1164,-844,1164,-2856,2164,-1852,1160,-2856,1156,-848,3172,-1852,1164,-840,2172,-848,1160,-1852,1160,-2852,2168,-1852,2164,-2856,2172,-844,6192,-2848,1164,-2852,1136,-1872,1164,-848,2144,-868,1168,-2848,1164,-844,3176,-1848,1168,-840,5192,-844,6196,-1844,4188,-840,2172,-844,6176,-1872,1164,-1848,3180,-1844,1168,-3852,1164,-1844,1168,-2848,2176,-1840,2176,-2848,1164,-1844,2180,-832,1176,-832,1176,-2840,2180,-836,3188,-832,6204,-1840,1196,-1812,1204,-804,1176,-1836,2180,-832,1180,-832,1200,-2816,2208,-1808,1204,-2812,2208,-1808,1200,-1808,4224,-832,1180,-1804,2212,-2808,1204,-1808,2208,-836,1172

Just by looking at the differences in marks in spaces timings it looks like there is a drift in timing that slowly adds up. Eventually this leads to one bit being mis-read by the sensor and everything after that point is decoded incorrectly.

I could also be interpreting this incorrectly though.

[greenmikey]

You probably won't love my solution but because the infoglobe protocol does not have both spaces and marks to represent a single bit I think I lose a bit of the wiggle room that normal protocols afford. It is very likely that I could also be misunderstanding the best way to use this library but as I said before this is my first project using c/c++/arduino (or whatever it should be called). After spending a bit of time on my lunch break today reading through it looks to be very well thought out and leaves room for implementing all sorts of protocols. I will try to implement everything using sendGeneric over the next few days.

Changes that worked - only the send function changed and I just adjusted the length of the marks and spaces to allow for fluctuations in timing based on our goal of 1ms minus time elapsed. It seems solid for now

void sendHexRaw(unsigned char *sigArray, unsigned int sizeArray, unsigned int khz) {
  uint32_t sigTime = micros(); //time at start
  uint32_t delayTime; 
  irsend.enableIROut(khz,33);
  for (unsigned int i = 0; i < sizeArray; i++) { //iterate thru each byte in sigArray
    register uint8_t bitMask =  0x80; //starting value of bitmask fo each Hex byte

    while (bitMask) { //do 8 times for each bit of the 8 bit byte
      sigTime += 1000; //Time 1ms after our last operation (or start)
      delayTime = sigTime - micros(); //The difference between current time and 1ms after our last bit
      if (bitMask & sigArray[i]) { //its a One bit
        irsend.space(delayTime); // LED off for 1000 usec = 1msec
        //Serial.print('1');  //for debug - ruins timing but ensures you use the correct bits
      }
      else { // its a Zero bit
        irsend.mark(delayTime);
        //Serial.print('0'); //for debug - ruins timing but ensures you use the correct bits
      }

      bitMask = (unsigned char) bitMask >> 1; // shift mask bit along until it reaches zero > exit the loop
    }
  }
}

https://www.youtube.com/watch?v=bu3tmA-2Q6M

Finally, what is the best way to send a bunch of the good reads I've collected? I have a full Analysir session from when I got it of good sends. I can post a collection of raw data or any other output format that you want.

[crankyoldgit]

I'm glad for you that you got it all working! Huzzah!

FYI, I've found that signal timings (e.g. trying to get it to exactly 1ms) have a large tolerance so you don't have to be exact. Oh it helps, and is admirable, and we strive for accuracy etc. But at the end of the day, because it's an analogue process (The IR signal detection) they build in a fairly large error margin, so you can be sort of slack. However, you have the device in front of you, so it's your call as to what works/doesn't.

Very minor nitpick: You timing method will have a problem when the micros() wraps, which happens every 71.58 minutes. Only if sending a message across that boundary of course. If you have a look at the IRtimer class, it tries to handle that problem.

As for the output format for the raw data to give us; A few captures using the IRrecvDumpV2 program will suffice. This is really a fairly simple protocol.
Realistically, just the uint16t rawData[XX] - {NNN, NNN, ..., NNN}; lines will do (with a description of what is being sent. e.g. 'Scrolling "Hello World!"' or "Non-scrolling: 'FooBar123'", "Blinking: "OMG Turn it off"' etc.)

Oh another tip. At the end of your "protocol" function, I'd include a LedOff() call at the end to ensure the LED is off as you technically could end with a mark signal. i.e. last bit being a '0' would/could leave the LED on.

There isn't really much need to port over to sendGeneric() It probably won't be as accurate as your method timing-wise, but as a lot of other protocols use it, it keeps the overall compiled-size of the library small by using common code. That is something we aim for as space is tight etc. Another advantage is any other improvements in sendGeneric and testing etc will get carried over automatically.

[crankyoldgit]

Re: The video. Hahaha!

[crankyoldgit]

Oh, just read your update in https://github.com/markszabo/IRremoteESP8266/issues/465#issuecomment-390525356

The only way to be sure (timing-wise) is to use a more-precise measuring method. e.g. An Oscilloscope.
Relying on the timing information from a capture on a micro-controller and an IR receiver module has a lot of added delays built in etc. e.g. Time spent in the interrupt handler; lag in the demodulation in the IR module. etc etc.
Also there is slop in how the speed of the IR LED being turned on/off, and the routines for toggling the GPIO, and the code/library etc.

TL;DR: You need something external and higher precision to find out where the slack is.

You could try adjusting the value of 1000 up/down depending for the mark, and the space function calls to see if you get a better/more reliable 1ms signal. e.g. mark(900) & space(1100) etc.

[greenmikey]

Oh, I didn't even think of the timer wrap! I will eventually implement that. I plan on making this a time/weather/custom message globe-thing with a web interface. I don't think it will be too bad in the end if I can squeeze it all in. I've already disabled all the protocols I'm not using to keep them from compiling. I'm not sure how much that helps with the cause though.

I'll have to dump the data from an arduino as my esp8266 is died up. I'll use IRremote's IRrecvDumpV2.ino. I hope that will be enough! I will get you 10 or so dumps of varying length data in a few days. I'll also get some more details about how data is formed written out and then close up the issue.

OH! irsend.ledOff(); is protected. I forgot that is why I was using that ugly code. I also see that I changed it to a digitalWrite(IR_LED, LOW) on my end. I put it outside of my protocol because I was not sure how slow it was (you said it was slow) and I didn't want to mess with the timing between header and message.

[crankyoldgit]

IRremote's _IRrecvDumpV2.ino_ only goes down to 50us granularity (we go down to 2us) and typically only handles <= 128 mark/space pairs. Depending on your message, you might exceed that.
But hey, it will do in a pinch and will make my life slightly easier. (Hmm, I should write a converter for analysir output to ours.)

Ah yes, it (LedOff()) is protected. But it isn't if its inside the class (hint hint, add it to the library proper ;-)
I'm fairly sure I effectively end all mark() calls with it anyway, so it's probably moot.
You probably want to enforce a gap after message anyway. e.g. space(100000); // 100msec will probably do to allow distinguishing between messages, so they don't run back-to-back.

You don't need to disable the (sending) protocols if you don't use them. The unused won't be included in the program size. It will however save you a few seconds of compile time per library recompile.
Receiving/decoding however is another story. Disabling there really makes a difference to speed and code size.

Re: ESP8266. Go on, order another one from ebay/aliexpress/banggood/etc ... they're cheap as chips (figuratively and literally ;-)

[greenmikey]

I don't know if pastebin links work around here but it was a bunch of text. I ran it on an ESP8266! The timings look weird. It must be the momentary delay between bits. I do know that it works pretty consistently. I'll have to make a video to show it off soon. I want to add a few more features though :) I've provided no examples of special transition effects. They work well on my side but I didn't want to muddy up anything and only included static and scrolling.

A quick note. This is from my implementation. I should really hook back up the real thing and repeat these tests. The issue is controlling the output. It has timed routines and a level of "randomness" in how they select text to display. It will be harder to get a number of samples without the transition effects. These transition effects are structured with two 0x00 bytes, the message in 8-bit ASCII hex, and finally a byte for which type of transition type.

https://pastebin.com/k7YiFnD9

Let me know if this does not work and I can paste in the message. This link may also help with figuring out what is going on. http://hanixdiy.blogspot.com/2010/10/hacking-infoglobe-part-3.html It seems to line up with what I've seen accurately.

Feel free to ask questions!

EDIT: I hooked back up the original setup. I pulled one message that I can control easily. Let me know if you want more of this type.

Timestamp : 000099.055
Encoding : UNKNOWN
Code : 6F103595 (32 bits)
Library : v2.4.1
Message: CID LIST
Transition: Static (0x01)

Raw Timing[63]:

• 7366, - 14, + 638, - 1356, + 12, - 630, + 4272, - 12,
• 1718, - 1320, + 12, - 664, + 2292, - 14, + 694, - 2360,
• 14, - 622, + 1366, - 14, + 618, - 3368, + 12, - 618,
• 4290, - 12, + 698, - 6298, + 12, - 686, + 2266, - 14,
• 1702, - 3284, + 12, - 722, + 2272, - 12, + 718, - 2366,
• 12, - 620, + 1366, - 12, + 620, - 1368, + 12, - 620,
• 2318, - 12, + 12, - 1646, + 1288, - 12, + 712, - 1330,
• 12, - 656, + 1290, - 14, + 694, - 2288, + 14

uint16_t rawData[63] = {7366, 14, 638, 1356, 12, 630, 4272, 12, 1718, 1320, 12, 664, 2292, 14, 694, 2360, 14, 622, 1366, 14, 618, 3368, 12, 618, 4290, 12, 698, 6298, 12, 686, 2266, 14, 1702, 3284, 12, 722, 2272, 12, 718, 2366, 12, 620, 1366, 12, 620, 1368, 12, 620, 2318, 12, 12, 1646, 1288, 12, 712, 1330, 12, 656, 1290, 14, 694, 2288, 14}; // UNKNOWN 6F103595

[crankyoldgit]

https://github.com/markszabo/IRremoteESP8266/wiki/Frequently-Asked-Questions#Im_getting_some_random_odd_values__50_usecs_in_my_capture_of_an_IR_message_What_is_up_with_that

Those 12 & 14 values look like a frequency mismatch .. or maybe the remote is to close to the receiver or something. They are not "normal".

And Yes, please get the "original" remote signals captured, not your implementation. We want make sure we are emulating the original, not a maybe-good copy.

Btw. The paste-bin worked fine.

[crankyoldgit]

I read that blog post. The frequency on the oscilloscope is 37600 Hz. You can use that, it's close enough to 38kHz that I don't think it will make any difference, but ya never know.
Also, the duty cycle is 50% reading the images of the oscilloscope. So you should change your line:
irsend.enableIROut(khz,33); to irsend.enableIROut(khz,50); or just irsend.enableIROut(37600); // Default is 50%

Are you sure your IR receiver is a 38kHz one? If it is, maybe a smoothing capacitor on the IR receiver module output?

[crankyoldgit]

Just chasing up where we are at with this issue. How goes it?

[greenmikey]

I'll have time to tinker later this week. It might just be a low quality sensor. I'm on a trip now but I'll try adding caps when I'll back. Thanks again for the support! Either way we can close it soon.

[crankyoldgit]

@greenmikey Cool. Just FYI, you should update to v2.4.2 of the library. It has some improvements with respect to timings on the frequency modulation.

[greenmikey]

Oh hell, I was not able to get clean results with a cap. I didn't spend much time trying to get it right though. I just saw your comment about using the newest version of the library. I'll look into it but I'll close this up and if I get good readings I'll contact to give results. Thanks for your help!