Sled: 32-bit ARM compatibility

[spacejam]

• [x] add test to make sure that the system never wraps by writing to an offset > 2^32
• [x] add additional concurrency verification tooling, as ARM has a weaker memory model than x86_64 (which makes it a great test platform in general)

[rubdos]

Maybe add ARM target to CI? I'm stumbling on a few bugs, so expect PR's.

[rubdos]

Okay, not sure how to solve... And I think this affects Intel 32 too:

Your Logical Sequence Number is a isize, but when serializing it, you assume it's 8 bytes/64 bits long.
Changing it to i64 is not that easy, since that'd require AtomicI64, which is currently nightly only. Please advice :-)

You can ping me on Gitter/IRC/slack/email/whatever for more direct communication, if you want me to test things out. I got a cross compiling environment setup for Android :-)

[spacejam]

@rubdos hey! sorry for the slow response, just started a new job and have been getting used to my new schedule. yeah, this is something I've been thinking a lot about. I think the correct solution here is to use double-wide LL/SC operations on 32-bit ARM architectures that support it, but this will require writing a little bit of C I believe for that target.

[rubdos]

There's no rush for me currently, but for me it's currently the blocker to port my application to Android! Not that I'm planning that anytime soon though. So take your time to start your new job; I wish you the best of luck!

--

I wonder though, aren't those ll/sc operations what AtomicI64 would use?

Is it, for you, an idea to have a nightly feature in sled, which uses AtomicI64 for the LSN? Then it's easily switched when AtomicI64 gets stabilized.

[spacejam]

@rubdos I would be OK with conditional compilation of a nightly-only feature for 32-bit targets, so long as 64-bit users can run it on stable.

[rubdos]

Roger that, I'll implement that the next few hours :-)

[rubdos]

Just wanted to drop this in here: I just started porting that application to Android. Currently running on x86_64 emulators/VMs, but soon testing on real ARM devices too. There's not a lot of load on sled for me, so I don't expect too funky things, but I'll let you know.

If in the meantime you want me to test something, feel free to drop me a note (here or on IRC/...)

[spacejam]

@rubdos that's super exciting, thanks for letting me know! At work we actually just got a batch of raspberry pi's, and I was thinking about running one in more or less a constant loop with the lock_free_delays feature set to get concurrency bugs to pop out. ARM is nice for this because of the weaker memory model, which should have a higher chance of causing bugs to pop out.

[rubdos]

That's cool too! If you want a bit faster ARM machine, you should consider the (aarch64) Odroid XU4. That thing is an octa core 64 bit ARM SBC! Might be useful too. I don't have one currently, but I'm planning to buy one for the same purpose: debugging concurrency bugs :-)

[spacejam]

@rubdos I just ordered one :D looking forward to trying to get this thing melt hahaha

[rubdos]

Nice, good luck melting it. I am quite confident that you won't succeed ;-)

[rubdos]

FYI: I had a few crashes on an x86 Android emulator (not even ARM), so seemingly some troubles with 32 bit devices.

[rubdos]

That's cool too! If you want a bit faster ARM machine, you should consider the (aarch64) Odroid XU4. That thing is an octa core 64 bit ARM SBC! Might be useful too. I don't have one currently, but I'm planning to buy one for the same purpose: debugging concurrency bugs :-)

Woops, I thought the XU4 was already 64 bit, but it's the C2 that's aarch64. It's still a way overpowered ARM device though; sorry for the confusion.

[rubdos]

This may interest you: https://aws.amazon.com/ec2/instance-types/a1/

[rubdos]

Heads up for those few people using this crate on 32-bit ARM: Rust team seems to have removed AtomicI64 from --target=arm-linux-androideabi, but not from --target=armv7-linux-androideabi.

[scriptjunkie]

Just to help anybody else who got here while trying to figure out how to compile sled for 32-bit architectures; the warning message says to use the nightly compiler, but that is not enough! I did
rustup install nightly-x86_64-unknown-linux-gnu
then
rustup default nightly-x86_64-unknown-linux-gnu
which you can verify by running
rustc --version
then to set the 32-bit target:
rustup target add armv7-linux-androideabi
and after downloading the ndk, adding the following to your ~/.cargo/config:

[target.armv7-linux-androideabi]
ar = "android-ndk-r19/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-ar"
linker = "android-ndk-r19/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi27-clang"

and to build your target:
cargo build -vv --target=armv7-linux-androideabi
which is normally enough, but if you depend on sled, it will still throw the same error until you edit your Cargo.toml to say:

[dependencies]
sled = { version = "0.19", features = ["nightly"] }

instead of just sled = "0.19"

[rubdos]

On 3/22/19 3:42 PM, Matt Weeks wrote:> Just to help anybody else who got
here while trying to figure out how to compile sled for 32-bit
architectures; the warning message says to use the nightly compiler, but
that is not enough! I did

rustup install nightly-x86_64-unknown-linux-gnu
then
rustup default nightly-x86_64-unknown-linux-gnu
which you can verify by running
rustc --version
then to set the 32-bit target:
rustup target add armv7-linux-androideabi
and after downloading the ndk, adding the following to your
~/.cargo/config:

[target.armv7-linux-androideabi]
ar =
"android-ndk-r19/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-ar"
linker =
"android-ndk-r19/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi27-clang"

and to build your target:
cargo build -vv --target=armv7-linux-androideabi

This is all very system-specific.

which is normally enough, but if you depend on sled, it will still
throw the same error until you edit your Cargo.toml to say:

[dependencies]
sled = { version = "0.19", features = ["nightly"] }

instead of just sled = "0.19"

We should add this to the warning message: "Only on a nightly compiler
with the `nightly' feature".

[spacejam]

@rubdos agreed, https://github.com/spacejam/sled/pull/596

[spacejam]

https://github.com/spacejam/sled/pull/602 removes nightly feature when we have access to the atomic*64 types

[spacejam]

https://github.com/spacejam/sled/pull/603 fixes the build on systems that don't have compatibility with the fs2 crate

[spacejam]

Now that we have the above things merged in, I think this is more or less settled. Please open another issue if something breaks or otherwise doesn't work on stable!