Arduino: Update ESP.getChipId() to take new OTP bytes into account

[whyameye]

I read in #921: "ESP sets its MAC address as a concatenation of 3 fixed bytes and 3 unique bytes." I think you mean that the 1st 3 bytes of the MAC address are the same between all 8266s. However, looking at the MAC addresses of my 12Es, some have 1st 3 bytes 18:FE:34 and some have first 3 bytes are 5C:CF:7F. Moreover I just purchased a 12F and its 1st 3 bytes are 60:01:94.

I'm concerned about this because I have been using getChipId as a unique identifier for all my 8266s. But now that I see that the 1st 3 bytes of the MAC address are perhaps not fixed after all, I wonder if it is possible that the last 3 bytes may not be unique between different runs of 12Es or between the 12E and 12F.

[kaeferfreund]

For this purpose I use the complete macadress as an id (with the separators removed). For now all have been unique.

[Humancell]

Yes ... you really MUST use the entire MAC address to ensure uniqueness. It's quite common that a vendor ID (the first three bytes) might run out ... it's only good for 16,777,215 unique IDs or less. A vendor will then go obtain a new vendor ID to use.

[igrr]

There are now more OTP bytes used to set MAC address because we have run out of space with three bytes. getChipID doesn't take these new bytes into account, it has to be updated.

Edit: using MAC address as unique ID a good approach for now.

[d-a-v]

This is a two years old thread tagged with milestone 2.5.0.
The issue is more true today with the number of esp8266s out in this world.
ESP.getChipId() is 32 bits.
We can change it to be 64 bits to return the full 48bits mac address.

[whyameye]

Ok but I wonder if at this point it would be best if we had backward compatibility on this, so maybe a bool flag that maybe defaults to the 32 bit version would be good?

[Makuna]

Is there any reason that we could not just add one that returns a 64 bit value?
Further, using the MAC address directly has complications if the sketch also changes the mac address.

There is a register for chipid, but it doesn't seem to be used for the ESP.getChipId() aka system_get_chip_id();

It would be nice to get a few samples from people who run the following simple sketch and copy their captured serial data. From my esp8266, I can see no relationship between the unmodified mac address upper bytes and any of these registers.

@igrr I also wonder why the chipid register is not used for the actual chipid?

#include <ESP8266WiFi.h>

void setup() {
  Serial.begin(115200);
  while (!Serial);
  Serial.println();

  uint32_t mac0 = MAC0;
  uint32_t mac1 = MAC1;
  uint32_t chipid = CHIPID;
  uint32_t apiChipId = ESP.getChipId();
  String macAddress = WiFi.macAddress();

  Serial.print("mac0 = ");
  Serial.println(mac0,HEX);
  Serial.print("mac1 = ");
  Serial.println(mac1, HEX);

  Serial.print("chipid = ");
  Serial.println(chipid, HEX);
  Serial.print("apiChipId = ");
  Serial.println(apiChipId, HEX);

  Serial.print("macaddress = ");
  Serial.println(macAddress);
}

void loop() {}

[d-a-v]

mac0 and chipid seem to not have much "entropy"
mac1 seems to be (chipid >> 8)|0x02000000

if the sketch also changes the mac address

Mac Address is hardcoded and can only be changed at runtime right ?
We could backup the STA ethernet address on boot.

mac0 =       0xc7840000
mac1 =       0x0200d1ce
chipid =     0x5a00b000
apiChipId =  0x00d1cec7
macaddress = 18:FE:34:D1:CE:C7

mac0 =       0x51500000
mac1 =       0x0200c3ad
chipid =     0x6400b000
apiChipId =  0x00c3ad51
macaddress = 5C:CF:7F:C3:AD:51

#include <ESP8266WiFi.h>
void setup() {
  Serial.begin(115200);
  Serial.println();
  uint32_t mac0 = MAC0;
  uint32_t mac1 = MAC1;
  uint32_t chipid = CHIPID;
  uint32_t apiChipId = ESP.getChipId();
  String macAddress = WiFi.macAddress();
  Serial.printf("mac0 =       0x%08x\n", mac0);
  Serial.printf("mac1 =       0x%08x\n", mac1);
  Serial.printf("chipid =     0x%08x\n", chipid);
  Serial.printf("apiChipId =  0x%08x\n", apiChipId);
  Serial.printf("macaddress = %s\n", macAddress.c_str());
}
void loop() {}

[Makuna]

The current Mac address is composed of parts of MAC0 & MAC1 (lower three bytes) and some magic numbers for the upper three bytes. The upper three bytes seem be to be one of a small set of static values (0x18fe34, 0xacd074, 0x600194, 0x5ccf7f, ?) across all esp8266.

mac address lower three bytes = ((MAC1 & 0x0000ffff) << 8) | (MAC0 & 0xFF000000) >>24)
These same ones are used for the chip id api.

There seems to be at least one byte in MAC0 that is unique but not used, (MAC0 & 0x00FF0000).

I have seen code that uses the third byte (MAC1 & 0x00FF0000) to switch which of the static sets of values are used for the upper part of the mac address. But I am pretty sure it was wrong.

Looks like the chipid register isn't very unique.

Wemos D1:
mac0 = 0x466F0000
mac1 = 0x0200100A
chipid = 0x1700B000
apiChipId = 0x00100A46
macaddress = 60:01:94 : 10:0A:46

Node MCU:
mac0 = 0xF5840000
mac1 = 0x0200F90F
chipid = 0x0000A000
apiChipId = 0x00F90FF5
macaddress = 18:FE:34 : F9:0F:F5

[d-a-v]

So what about

• using ((mac0<<40) | ((chipid>>24)<<56) | macAddress)
  your two examples would give 6F:17:60:01:94:10:0A:46 and 84:00:18:FE:34:F9:0F:F5
• prepare it before users have a chance to live-change mac address
• api: uint64_t ESP.getChipId64()

[4Source]

esp8266 core v2.5.2

ESP-12F
mac0 = C4EB0000
mac1 = 200746E
chipid = 6400B000
apiChipId = 746EC4
macaddress = 5C:CF:7F:74:6E:C4

another ESP-12F
mac0 = C7870000
mac1 = 200746D
chipid = 6400B000
apiChipId = 746DC7
macaddress = 5C:CF:7F:74:6D:C7

if that is still needed.

[devyte]

I think this requires further discussion and thought.
Pushing back

[d-a-v]

A step further in this discussion
(this issue is marked for 2.7.0).

apichipid = ESP.getChipId() is part of the mac address so we shouldn't use both together (edited)

There are three paths, let's :smile: for the first, :tada: for the second, :heart: for the third and :-1: if disagree with all of them.
Edit: :+1:, :rocket: and :eyes: added

• :smile: Xor all relevant numbers:
  mac0, mac1 and chipid are 32 bits numbers. We can push them to the right(the other right) (<<32) and xor everything also with the mac address. Given the examples above, these numbers are not always the same, and fitting them in 64 bits is probably the best we could do at low cost:
  (mac0<<32) ^ (mac1<<32) ^ (chipid<<32) ^ macAddress
((mac0 ^ mac1 ^ chipid) << 32) ^ macAddress

• :tada: CRC64:
  The best would be to make 64 bits CRC out of all of them (18 bytes):
  (mac0<<(8*(4+4+6))) | (mac1<<(8*(4+6))) | (chipid<<(8*(6))) | macAddress
  It comes with a cost: a 64 bits crc function is needed

• :heart: CRC32:
  Like the above but compute a CRC32 of the 18 bytes We already have the crc32 function in flash anyway so it comes at a little smaller cost than the second with a better entropy than the first. Still 32bits though.

edit:

• :+1: Backward compatible with ESP.getChipID() (using & 0xffffff)
  (mac0 ^ mac1 ^ chipid) << 24 | apiChipId
  as suggested by @dirkmueller in the following comment

  this allows somewhat of a backward compatbility (new value & 0xffffff is the old chipid) which might be important for upgrade (when the chipid is used to match nodes)

• :rocket: Backward compatible with ESP.getChipID() (using & 0xffffffff or static_cast<uint32_t>)), with added crc32 entropy
  It is CRC32 above mixed with apiChipId (4th byte will always be 0)
  (CRC32 << 32) | apiChipId

• :eyes: Simple and enough, also compatible with ESP.getChipID() (using & 0xffffffff orstatic_cast<uint32_t>))
  Initial extension proposal: use full mac address which is supposed to be unique in the world
  first three bytes are lower mac address (= ESP.getChipID())
  last three bytes in the upper 32 bit value
  With mac=aa:bb:cc:dd:ee:ff:

  • ESP.getChipID() == 0xddeeff
  • ESP.getChipID64() == 0x00aabbcc00ddeeff

[dirkmueller]

what about (mac0 ^ mac1 ^ mac2) << 24 | chipid ? this allows somewhat of a backward compatbility (new value & 0xffffff is the old chipid) which might be important for upgrade (when the chipid is used to match nodes)

[d-a-v]

Vote message updated, voters are needed :)

@dirkmueller chipid is 32 bits no ? should it be <<32 not <<24 ?

[dirkmueller]

@d-a-v in all esp8266 that I have available, chipid is simply the serial portion triplet of the STA mac address (not the vendor portion which are the first 3 bytes). So it fits into 24 bit, and there are 8 bit still available for deduplication. So we can do something like xor or crc32 over the first triplet of the mac

[d-a-v]

What you call chipid is mentioned as ESP.getChipId() above.
There is another CHIPID (4 bytes long), check above posts.
Vote message is updated accordingly.

[devyte]

As explained in the first comments, the MAC address is 6 bytes (48bits) long, and its default value is unique. It is made up of 3 bytes ID assigned to a vendor (a vendor can have multiple of these), and 3 bytes for individual devices manufactured, unique for a particular ID.
The MAC can be changed from its default value by the user, but the default can be cached before changing. That means that the default MAC can be used to generate a unique chipId at 48bits (i.e.: contained in a 64bit integer).
Adding additional bits beyond the 48bits as a function of the 6 MAC bytes (e.g.: crc or whatever) doesn't make sense, because it doesn't add entropy. It's also not necessary, given that the default MAC is already unique.
It is not possible to assure a unique chipId at 32bits based on any combination of the MAC bytes. No matter what is done, there is a small chance of a chipId clash between 2 or more ESPs.
The only way to assure a unique 32bit chipId would be with an external serial number chip that provides a unique 32bit number, and such a set up would be up to the user.

My view on this is to change the return type of ESPClass::getChipId() to be uint64, return the default MAC, and put this issue to rest. That would be a breaking change due to the type change, so target for v3.

Side Note: the code that generates the default SSID of ESP-XYZNUM would likely need to be checked.

[dirkmueller]

@devyte in general it is not possible to fit 48 bits into 32 bits without some small chance of collision. However here we're talking about esp8266 which is produced by a single vendor which has a very limited number of vendor prefixes assigned. Unless that number of prefixes assigned exceeds 8 bit, it is possible to generate a unique id in 32 bit.

Your suggestion is basically just 'stop using this, use the Mac' which users can do already today right now.

For the unfortunate group of users that used the chipid for something meaningful and that was stored in 32 bit fields in e.g. databases before it might be a good idea to offer a transition to a newer format that does not extend data store requirements.

My suggestion allows for that, a way to map from old to new format (that is reasonable collision safe given the limited number of vendor prefixes that can occur) and not extend storage requirements.