Rustup: Unable to install rustc on Raspberry PI W due to low memory

[faern]

I just tried to update to Rust 1.44.0 on my Raspberry pi zero W (512 MiB of memory). I was previously running 1.43.1 without problems. But during the upgrade I see:

info: installing component 'rustc'
 52.1 MiB /  69.1 MiB ( 75 %)   1.2 MiB/s in  1m 14s ETA: 14smemory allocation of 148512752 bytes failedAborted

I don't even run X on this thing, so before starting the install/upgrade I only use 36 MiB of my memory. I can't really free up much more than that.

rustup install 1.43.1 works fine. So it must be 1.44 that passes some magical limit.

Possible Solution(s)
Don't store the entire tarball in memory (if that is what actually happens) stream them directly to disk while downloading.

Notes

Output of rustup --version:

rustup 1.21.1 (7832b2ebe 2019-12-20)

Output of rustup show:

Default host: arm-unknown-linux-gnueabihf
rustup home:  /home/pi/.rustup

installed toolchains
--------------------

stable-arm-unknown-linux-gnueabihf (default)
nightly-arm-unknown-linux-gnueabihf
1.43.1-arm-unknown-linux-gnueabihf

active toolchain
----------------

stable-arm-unknown-linux-gnueabihf (default)
(rustc does not exist)

[kinnison]

You could try setting the max unpack ram lower which should limit how much allocation rustup does. We are working on making that slightly more automatic. If you set RUSTUP_UNPACK_RAM=250000000 in your environment that ought to limit rustup to 250M of RAM.

[tesuji]

Can rustup read the available free RAM memory and decide how much memory it need?

[faern]

250MB also failed for me. But RUSTUP_UNPACK_RAM=200000000 worked. And below 200MB was not allowed to set. So it seems there is a very narrow span where it actually works.

It feels like an allocation error should not abort, but rather fall back to disk based unpacking? That feels more stable than trying to guess how much RAM it can allocate. And if there is such a mode, it would be nice if it could be activated with RUSTUP_UNPACK_RAM=0 or something.

[faern]

There seems to be a discrepancy in how RUSTUP_UNPACK_RAM should be used and how it's documented. The README says the minimum is 100M. But when I run with RUSTUP_UNPACK_RAM=100000000 rustup install 1.44.0 I get:

thread 'main' panicked at 'RUSTUP_UNPACK_RAM must be larger than 200000000', src/dist/component/package.rs:180:13

And the error says larger than 200000000 but in reality 200000000 works fine. So I get three mixed messages here.

[rbtcollins]

I think the README is stale; sorry. The minimum with threaded IO, is the
size of any one single file. We don't have a chunking implementation (yet)
for threaded IO.

The next release of rustup will require 220M, because the single largest
file size has crept up again; adding in a chunking implementation is a
pretty high priority step I think.

-Rob

[faern]

If the minimum requirement continues to creep up. And a 512MiB machine fails to run rustup with the allocation set to 250MB. Then we are pretty close to not having any working value for RUSTUP_UNPACK_RAM at all on these machines. So automatically detecting it won't work well in all cases I suspect?

I have tested and can confirm that using RUSTUP_IO_THREADS=disabled rustup install 1.44.0 works. It feels like it should fall back to this or something else if allocation fails, instead of erroring out.

EDIT: The RPi zero W is single threaded. Why does disabling threading works different if the default is to use the number of CPU cores? I have not checked what the underlying CPU count library returns on this machine, but I suspect 1. And if so, should that not make it use the same mode as manually disabling threading?

[rbtcollins]

The unthreaded code path is very different to the threaded code path; putting a check for thread-count=1 and switching to unthreaded would be a very useful enhancement, if you'd like to submit that soon we could include it in the next release.

[sk-Prime]

memory allocation error happened when i tried to update from 1.44 to 1.45 in raspberry pi zero. uninstalling rust and installing again using curl produces no error. rustup version 1.21

RUSTUP_UNPACK_RAM=100000000 rustup update worked too

[Harvzor]

My distro didn't have any swap enabled, so I installed zram using this guide: https://pimylifeup.com/raspberry-pi-zram/

It's increased the amount of RAM I have available allowing it to install Rust.

[nekevss]

Following up. I had to install zram and then run the below command to install rust (mainly get past the second rustc download).

RUSTUP_UNPACK_RAM=16777216 rustup toolchain install stable-armv7-unknown-linux-gnueabihf

[kinnison]

@nekevss Did you need that even with Rustup 1.24.1 ?

[nekevss]

@kinnison Yepp, rustup 1.24.1 with rustc 1.51.0.

Worth noting, I just got my Raspberry Pi 4, so I could've been overlooking something. But I kept getting the error "memory allocation of 16777216 bytes failed" on the second install/download of rustc. The above command finally got it to run and so I thought I'd add to the thread since this was helpful for me to figure out.