Etcher: OS X: Increased CPU usage in drivelist perl regular expression

[qkboo]

• Etcher version:
  1.0.0 beta10

• Operating system and architecture:
  Mac X 10.12.4

• Do you see any meaningful error information on DevTools?
  Nope!
  

I used this tool, serveral days, very interesting and useful.
But, after writing some image file, raspbian zipped image.
etcher remains some 'perl' processes like below.

---

perl -n -e/\((\d+)\sBytes\)/ && print $1

and I followed this process then found more

~$ ps -ef |grep 21296
  501 21296 21292   0 화05AM ??       878:52.60 perl -n -e/\((\d+)\sBytes\)/ && print $1

~$ ps -ef|grep 21292
  501 21292 21196   0 화05AM ??         0:00.00 /bin/bash /var/folders/mn/prcr5p_95vx_m1rx342svyvc0000gn/T/.io.resin.etcher.G7bxhO
  501 21296 21292   0 화05AM ??       879:35.17 perl -n -e/\((\d+)\sBytes\)/ && print $1

@ ~$ ps -ef |grep 21196
  501 21196     1   0 화05AM ??         0:00.01 /bin/bash /var/folders/mn/prcr5p_95vx_m1rx342svyvc0000gn/T/.io.resin.etcher.G7bxhO
  501 21292 21196   0 화05AM ??         0:00.00 /bin/bash /var/folders/mn/prcr5p_95vx_m1rx342svyvc0000gn/T/.io.resin.etcher.G7bxhO

it did not sleep my computer and processing something... i don't know
it make overheat my macbook, all time 60~65 degree and run fans through 3500~4000 rpms.

when i kill theses process my computer back to normal condition and very cool.

I will try 1.0.0 rc2 and check and monitor this happend.
thanks.

[Pflanzgurke]

Had the same issue with 1.0.0 rc2 on the same OS version, however the perl command line looked different... didn't copy it, sorry it was perl -n -em .... or similar.

[lurch]

@jviotti Looks like this is coming from https://github.com/resin-io-modules/drivelist/blob/master/scripts/darwin.sh#L39 ? Perhaps there's been a slight change to the diskutil output that is now making this fail?
(BTW is perl guaranteed to always be installed on OSX?)

[jviotti]

Oh, interesting. Looks like we should either tweak that regular expression, or replace perl with something else.

(BTW is perl guaranteed to always be installed on OSX?)

Yeah, it should come pre-installed in all OS X systems.

@lurch What do you think about this? Can we perform this task in some other less resource intensive way?

What we need is the following: diskutil info /dev/diskN output a line like this:

   Disk Size:                141.0 GB (140999917568 Bytes) (exactly 275390464 512-Byte-Units)

And we want to get the 140999917568 number from that string.

[lurch]

Well, assuming the line is always formatted like that, a pretty crude solution would be piping it to something like | cut -d '(' -f 2 | cut -d ' ' -f 1 :-)
But do we know why the perl regex is failing / using up lots of CPU? Obviously whatever's causing that to fail might cause my cut solution to fail too.

Sounds like @Pflanzgurke 's problem was caused by https://github.com/resin-io-modules/drivelist/blob/master/scripts/darwin.sh#L41 ?

Just for dev-purposes, can we get example diskutil and mount output for all versions of OSX that we support, in case there's subtle differences?

[jviotti]

But do we know why the perl regex is failing / using up lots of CPU? Obviously whatever's causing that to fail might cause my cut solution to fail too.

I was initially considering that regex could be trying to scan multi line patterns, therefore forcing the regex evaluator to act on the whole file, however that doesn't seem to be the case, given that perl just acts on the line I sent above (after a grep).

It could be regex issue but I can't say for sure. @qkboo What macOS version are you running? Maybe worth making @qkboo test various alternatives to measure them in his own system?

Just for dev-purposes, can we get example diskutil and mount output for all versions of OSX that we support, in case there's subtle differences?

That sounds really tricky. Its not easy to get VMs for macOS versions other than the one running in your computer at that time.

[qkboo]

@jviotti I used macOS siera 10.12.4. And I can test various alternatives for worth making.

[lurch]

Thanks for your offer of help @qkboo - here's some things you can try for us:

• Report the output of both diskutil list and diskutil info /dev/diskN (where diskN is whatever drive your SD card appears as)
• Report the ouput of perl --version
• Try downloading and running https://github.com/resin-io-modules/drivelist/blob/master/scripts/darwin.sh in a terminal, and see if that still locks up with 100% CPU?

[lurch]

And @Pflanzgurke if you could do the same too, that'd be really helpful :-)

[Pflanzgurke]

$ perl --version

This is perl 5, version 18, subversion 2 (v5.18.2) built for darwin-thread-multi-2level
(with 2 registered patches, see perl -V for more detail)

---

$ diskutil list
/dev/disk0 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *500.3 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_CoreStorage Macintosh HD 499.4 GB disk0s2
3: Apple_Boot Recovery HD 650.0 MB disk0s3

/dev/disk1 (internal, virtual):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_HFS Macintosh HD +499.1 GB disk1
Logical Volume on disk0s2
ED605CD9-6EBC-49D1-A205-FB7F5CAD93E8
Unencrypted

/dev/disk2 (disk image):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_partition_scheme +20.0 MB disk2
1: Apple_partition_map 32.3 KB disk2s1
2: Apple_HFS Flash Player 19.9 MB disk2s2

/dev/disk3 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_partition_scheme *2.0 GB disk3
1: Apple_partition_map 4.1 KB disk3s1
2: Apple_HFS 2.4 MB disk3s2

---

$ diskutil info /dev/disk3
Device Identifier: disk3
Device Node: /dev/disk3
Whole: Yes
Part of Whole: disk3
Device / Media Name: Mass-Storage

Volume Name: Not applicable (no file system)
Mounted: Not applicable (no file system)
File System: None

Content (IOContent): Apple_partition_scheme
OS Can Be Installed: No
Media Type: Generic
Protocol: USB
SMART Status: Not Supported

Disk Size: 2.0 GB (1971322880 Bytes) (exactly 3850240 512-Byte-Units)
Device Block Size: 512 Bytes

Read-Only Media: No
Read-Only Volume: Not applicable (no file system)

Device Location: External
Removable Media: Removable
Media Removal: Software-Activated

Virtual: No
OS 9 Drivers: No
Low Level Format: Not supported

---

$ ./darwin.sh
device: /dev/disk0
description: "APPLE SSD SM0512G"
size: 500277790720
mountpoints: []
raw: /dev/rdisk0
protected: False
system: True

device: /dev/disk1
description: "Macintosh HD"
size: 499055067136
mountpoints:

• path: "/"
  raw: /dev/rdisk1
  protected: False
  system: True

device: /dev/disk3
description: "Mass-Storage"
size: 1971322880
mountpoints: []
raw: /dev/rdisk3
protected: False
system: False

I do not see the problem when running it straight from the terminal, but who knows what triggered the excessive CPU utilization last time...

BTW, thanks for a great tool!

[lurch]

Thanks for the info @Pflanzgurke :+1:

I do not see the problem when running it straight from the terminal

Hmmm, very strange. Do you still see excessive CPU usage when running Etcher itself?

[Pflanzgurke]

I flashed a USB stick with Etcher again, just as I did before (but I have updated to rc3 sometime before) and could not reproduce the bug. If it happens again, I will try to do more diagnosis. Not sure what happened when the script went wild...

[jviotti]

I just experienced this issue. Perl was taking a decent amount of CPU on my MacBook Pro, even though I had closed Etcher long ago. I can't manage to reproduce again neither :/

[lurch]

Sounds like maybe Perl missed the EOF notification, and is busy-looping waiting for more input? (just a wild (probably wrong) guess)

[hvtuananh]

My old MacBook Air is running very slowly after running Etcher. I have no idea until today, when I look at a very "suspicious" perl script that consume 100% CPU constantly:

perl -n -e'm{^'"${disk}"'(s[0-9]+)? on (.*) \(.*\)$} && print "$2\n"'

[hvtuananh]

Also, it seems to create about 60 shell scripts, all with the same content over the course of 2 minutes after launch:

-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:45 tmp-34923aEHTf6VAGQh9.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:45 tmp-34923srutlapWc095.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:45 tmp-3492344iZV4RjOtlw.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:45 tmp-34923DRKh5iC3bkA5.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:45 tmp-3492337bIQGPV4XT6.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:45 tmp-34923apuZ2Yv6opMq.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:45 tmp-34923IhFgBCjUx0ym.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923syav4SkWFmjq.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923KdniUxsSCjuS.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923yV8JQJVr7xLk.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923VIsXpKj32cqG.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923setVcGuGPrHF.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923CEWyzEiJXoR1.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923wLpjPD0M6x4G.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923TECIVEtGv5n3.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923Ha921MzBgxPv.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923Nn9CxH8MMiIY.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923176lWB8zTli7.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923c8VkfSslEg5l.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923r35r4huwkotn.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923CMsfXytS9NGB.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923CPSXHlcoDQvu.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-349233naaPWu8Kran.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923fRDDKRwMMOyu.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-349234bWYVeKNY9a5.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923R56FmaHwQN6l.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923vCoKOdi69Qpi.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923jOOkA83s4T7M.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-349237x0RcGncYnty.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923orhQ571deAzU.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-349237lQ7qejz6kd7.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923D5ngShEL29Ds.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923lyNApoOSe6Qp.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923JjvQcK3AD27w.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923Nrwl7Jv0y5K4.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-34923qSaytRXMnRhn.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:46 tmp-3492318NUSTiv7iDi.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923FtdpffAGKOvd.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923FlQDt4otcDhq.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923CtEfxcJ6pSsb.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-349234JpYIJziIcww.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923bT48Dskxjugt.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-349239ZzlU7ydetJf.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923qqX43SbKzist.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923t8QoNu0JoiYI.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-349238whkRzk5eitH.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923l0uD1a3bqx5D.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923sv75ObD7SikG.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923WPheemZHUofz.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-349239k41pIGWXau4.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923GDOj0aivSnnk.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923NtKvh5CKQFI3.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923vzaoT7NIsVyC.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-349234sgqjEenwfWg.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923ZZlCJ2JdiKeT.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:47 tmp-34923N2FvHAGe04FO.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:55 tmp-34923q3W73QcBMWFs.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:55 tmp-34923FOMjB9xtRhFv.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:55 tmp-349238M2ET5HKrfO5.sh
-rwxr-xr-x  1 tuananh  staff  2320 Jun  3 20:55 tmp-34923XyVt32ygc44b.sh

[lekkas]

Just hit this as well, CPU usage went up to 99% (Etcher version v1.1.0)

[edit] Forgot to mention, the perl script appeared to keep running after I had exited Etcher.

[lurch]

Re-opening issue, until drivelist gets bumped in Etcher.

[lurch]

For anyone still suffering from this issue, we believe that https://resin-nightly-downloads.s3.amazonaws.com/etcher/2017-06-08/1.0.0%2B8811bbe/Etcher-1.0.0%2B8811bbe-darwin-x64.dmg should solve it.

[pixeltrix]

FYI, I just had to kill an Etcher perl process:

perl -n -em{^/dev/disk0(s[0-9]+)? on (.*) \(.*\)$} && print "$2\n"

It was still consuming 99% of a CPU core many hours after I'd quit Etcher. This was on Mac OS Sierra 10.12.6 using Etcher 1.1.2. Seems like this is still a problem?

[jhermsmeier]

Oh man – thanks @pixeltrix for the heads-up! We'll get onto implementing that through native APIs, to avoid this completely.

[lurch]

implementing that through native APIs

See https://github.com/resin-io-modules/drivelist/pull/172

[poregon]

+1 @pixeltrix
Identical issue and setup as @pixeltrix (etcher 1.1.2; Sierra 10.12.6). Noticed this perl script running today after seeing battery life reduce from >4h to <1h. Had suspected it was associated to a virus, so thank you for the comment @pixeltrix.

[mraaroncruz]

This is still an issue. Run away perl process ages after closing app.
Etcher 1.2.1
macOS 10.12.6
Activity Monitor output

[lurch]

@mraaroncruz Yup, but the good news is that this will be fixed in the next release of Etcher (which is coming soon) - see https://github.com/resin-io-modules/drivelist/pull/238 and https://github.com/resin-io-modules/drivelist/pull/246

[jhermsmeier]

Fixed in #1975