Platformio-core: Add : espressif : make spiffs

[itsjustvenky]

In Arduino-esp8266 project there is a option to build and upload just SPIFFS image to ESP.

http://esp8266.github.io/Arduino/versions/2.0.0/doc/filesystem.html#uploading-files-to-file-system

Is it possible to add similar functionality to "PlatformIO" where I create a folder named "data" in the project root folder and set the flag/property to build just SPIFFS image and upload it to ESP using serial or ota

@igrr (and yourself) already provided mentioned the document the option to upload just SPIFFS image to ESP over OTA

https://github.com/esp8266/Arduino/blob/master/doc/platformio.md

Let me know.thanks.

[ivankravets]

@itsjustvenky please provide console output from Arduino IDE of this process. Thanks.

[ivankravets]

@igrr could you explain us how does it work? What we should implement?

Thanks.

[itsjustvenky]

@ivankravets The SPIFFS bin generation and upload to ESP is done a arduino IDE plugin jar file.

https://github.com/esp8266/arduino-esp8266fs-plugin/
http://esp8266.github.io/Arduino/versions/2.0.0/doc/filesystem.html#uploading-files-to-file-system

I am till figuring out the details on the SPIFFS bin file generation.

"igrr" is the correct person to answer your question. thanks.

[ivankravets]

Thanks, I see now how does it work https://github.com/esp8266/arduino-esp8266fs-plugin/blob/master/src/ESP8266FS.java

Will back to this issue as soon as possible.

[itsjustvenky]

I found this too (just reference)

https://github.com/pellepl/spiffs

[itsjustvenky]

Ok this what is the java (Processing app) is using

https://github.com/igrr/mkspiffs

[itsjustvenky]

The binary that is used to generate SPIFFS image is "mkspiffs" and it takes parameters from

(on Win7) C:\Program Files (x86)\Arduino\hardware\esp8266com\esp8266\boards.txt

example

generic.menu.FlashSize.4M1M=4M (1M SPIFFS)
generic.menu.FlashSize.4M1M.build.flash_size=4M
generic.menu.FlashSize.4M1M.build.flash_ld=eagle.flash.4m1m.ld
generic.menu.FlashSize.4M1M.build.spiffs_start=0x300000
generic.menu.FlashSize.4M1M.build.spiffs_end=0x3FB000
generic.menu.FlashSize.4M1M.build.spiffs_blocksize=8192
generic.menu.FlashSize.4M1M.build.spiffs_pagesize=256
generic.menu.FlashSize.4M1M.upload.maximum_size=1044464

this code is from the java file

 if(listenOnProcess(new String[]{toolPath, "-c", dataPath, "-p", spiPage+"", "-b", spiBlock+"", "-s", (spiEnd - spiStart)+"", imagePath}) != 0){

and then the image is getting uploaded into ESP using Esptool

sExec(new String[]{esptoolPath, "-cd", resetMethod, "-cb", uploadSpeed, "-cp", serialPort, "-ca", uploadAddress, "-cf", imagePath});

and the esptool parameters too are taken from "boards.txt" (depending on the board the user selected).

[0x2b3bfa0]

:+1:

@ivankravets: Practical example of @itsjustvenky post (BASH flavored):

#!/bin/bash

MKSPIFFS="/usr/bin/mkspiffs"
ESPTOOL="/usr/bin/esptool"

PROJECT_FOLDER="/home/ivan/myproject/"
TEMP_FILE=$(mktemp /tmp/spiffs_image-XXXXXXXX.bin)

SERIAL_PORT="/dev/ttyUSB0"
RESET_METHOD="nodemcu"
UPLOAD_SPEED=115200

SPIFFS_START_ADDRESS=0x300000
SPIFFS_END_ADDRESS=0x3FB000
SPIFFS_PAGE=256
SPIFFS_BLOCK=8192
SPIFFS_SIZE=$((SPIFFS_END_ADDRESS - SPIFFS_START_ADDRESS))

$MKSPIFFS -c "${PROJECT_FOLDER}/data" -p "$SPIFFS_PAGE" -b "$SPIFFS_BLOCK" -s "$SPIFFS_SIZE" "$TEMP_FILE"
$ESPTOOL -cd "$RESET_METHOD" -cb "$UPLOAD_SPEED" -cp "$SERIAL_PORT" -ca "$SPIFFS_START_ADDRESS" -cf "$TEMP_FILE"

Note 1: All these uppercase variables must be retrieved from PlatformIO.ini or whatever file that contains the device parameters.
Note 2: I don't know if mkspiffs is distributed along with the esp8266-arduino framework. If not, is hosted here: https://github.com/igrr/mkspiffs

[ivankravets]

Note 1: All these uppercase variables must be retrieved from PlatformIO.ini or whatever file that contains the device parameters.

If user needs to enter all these vars then he will use Make :) PlatformIO should simplify all these things. I'm working on it.

[0x2b3bfa0]

@ivankravets: :smile_cat: Thank you very much! This will be a very great Christmas present for many of us!

[ivankravets]

Guys, let's test it!

• https://github.com/platformio/platformio/commit/cae5f1f5532fc34cb541cb3aaed83c752389ef4e
• https://github.com/platformio/platformio/commit/2c4c3a12ea7dc1abbea5f8bc843c61a027a55ce3

Docs http://docs.platformio.org/en/latest/platforms/espressif.html#uploading-files-to-file-system-spiffs

P.S: Need to install the latest development version http://docs.platformio.org/en/latest/installation.html#d-development-version

[0x2b3bfa0]

@ivankravets: Thank you very much! I'll travel now, however I'll try later (in 6h or tomorrow).

This is the better :gift: for us!

[itsjustvenky]

@ivankravets I will test it sometime tomorrow and get back to you. Thank you.

[itsjustvenky]

Upload to FS seems to writing at wrong address

Uploading 262144 bytes from .pioenvs/esp01_1m/spiffs_image.bin to flash at 0x00000000
erasing flash
size: 040000 address: 000000

The address is correct when building spiffs images, but while writing it is overwriting x0000000

"/home/venkat/.platformio/packages/tool-esptool/esptool" -vv -cd ck -cb 115200 -cp /dev/ttyUSB0 -cf .pioenvs/esp01_1m/spiffs_image.bin -ca 0xbb000

This is from arduino ide

[SPIFFS] data   : G:\Projects\Arduino\Arduino-IDE\test\FSBrowser\data
[SPIFFS] size   : 256
[SPIFFS] page   : 256
[SPIFFS] block  : 4096
/edit.htm.gz
/favicon.ico
/graphs.js.gz
/index.htm
[SPIFFS] upload : C:\Users\VENKAT~1.VEN\AppData\Local\Temp\build5945408886832196661.tmp/FSBrowser.spiffs.bin
[SPIFFS] reset  : ck
[SPIFFS] port   : COM10
[SPIFFS] speed  : 115200
[SPIFFS] address: 0xBB000

Uploading 262144 bytes from C:\Users\VENKAT~1.VEN\AppData\Local\Temp\build5945408886832196661.tmp/FSBrowser.spiffs.bin to flash at 0x000BB000

Note : Please ignore the sizes as the files (spiffs) are different in "arduino ide" project and platformio project.

I used 1M ESP with this LD script : build_flags = -Wl,-Tesp8266.flash.1m256.ld

I am on
PlatformIO, version 2.7.0.dev6

[ivankravets]

@itsjustvenky I've fixed it. Try PlatformIO 2.7.0.dev7

[ivankravets]

@crushedice2000, @comino could you test it?

[0x2b3bfa0]

@ivankravets: It works for me! :+1: Thank you very much!

[georgkreimer]

I'm getting this error here:

▶ platformio run --target uploadfs

[Wed Dec 30 01:27:03 2015] Processing huzzah (platform: espressif, board: huzzah, framework: arduino, build_flags: -Wl,-Tesp8266.flash.4m.ld)

"/Users/georgkreimer/.platformio/packages/tool-mkspiffs/mkspiffs" -c data -p 256 -b 8192 -s 3125248 .pioenvs/huzzah/spiffs_image.bin
/index.html
(["uploadfs"], [".pioenvs/huzzah/spiffs_image.bin"])
Auto-detected UPLOAD_PORT/DISK: /dev/cu.wchusbserial1410
"/Users/georgkreimer/.platformio/packages/tool-esptool/esptool" -vv -cd nodemcu -cb 115200 -cp /dev/cu.wchusbserial1410 -ca 0x100000 -cf .pioenvs/huzzah/spiffs_image.bin
esptool v0.4.6 - (c) 2014 Ch. Klippel [email protected]
setting board to nodemcu
setting baudrate from 115200 to 115200
setting port from /dev/tty.usbserial to /dev/cu.wchusbserial1410
setting address from 0x00000000 to 0x00100000
espcomm_upload_file
stat .pioenvs/huzzah/spiffs_image.bin success
opening port /dev/cu.wchusbserial1410 at 115200
tcgetattr
tcsetattr
serial open
opening bootloader
resetting board
trying to connect
setting character timeout 0
done
setting character timeout 1
done
espcomm_send_command: sending command header
espcomm_send_command: sending command payload
trying to connect
setting character timeout 0
done
setting character timeout 1
done
scons: *** [uploadfs] Error 255
espcomm_send_command: sending command header
espcomm_send_command: sending command payload
espcomm_send_command: receiving 2 bytes of data
espcomm_send_command: receiving 2 bytes of data
espcomm_send_command: receiving 2 bytes of data
espcomm_send_command: receiving 2 bytes of data
espcomm_send_command: receiving 2 bytes of data
espcomm_send_command: receiving 2 bytes of data
espcomm_send_command: receiving 2 bytes of data
espcomm_send_command: receiving 2 bytes of data
espcomm_open
Uploading 3125248 bytes from .pioenvs/huzzah/spiffs_image.bin to flash at 0x00100000
erasing flash
size: 2fb000 address: 100000
first_sector_index: 256
total_sector_count: 763
head_sector_count: 16
adjusted_sector_count: 747
adjusted_size: 2eb000
espcomm_send_command: sending command header
espcomm_send_command: sending command payload
setting timeout 10000
setting character timeout 100
done
setting timeout 1
setting character timeout 1
done
warning: espcomm_send_command: didn't receive command response
warning: espcomm_send_command(FLASH_DOWNLOAD_BEGIN) failed
closing bootloader
============================================================================= [ ERROR ] Took 10.85 seconds =============================================================================

[ivankravets]

Could you try for test to upload SPIFFS image using Arduino IDE? https://github.com/esp8266/Arduino/blob/master/doc/filesystem.md#flash-layout

Thanks

[georgkreimer]

Looks like that doesn't work either. Hmm, might be my cheap USB UART (CH340) that has only RX, TX, VCC and GND connected. Though uploading the sketch works fine.

Sketch uses 249,920 bytes (23%) of program storage space. Maximum is 1,044,464 bytes.
Global variables use 39,798 bytes (48%) of dynamic memory, leaving 42,122 bytes for local variables. Maximum is 81,920 bytes.
Uploading 254064 bytes from /var/folders/ky/fpk2bmgn4b30zjhyntzgtzhh0000gn/T/build3fadfb4cba027a61b568c284b4c3fb34.tmp/CaptivePortal.ino.bin to flash at 0x00000000
.........................................................................................................................................................................................................................................................
[SPIFFS] data : /Users/georgkreimer/Library/Arduino15/packages/esp8266/hardware/esp8266/2.0.0-rc2/libraries/DNSServer/examples/CaptivePortal/data
[SPIFFS] size : 3052
[SPIFFS] page : 256
[SPIFFS] block : 8192
/index.html
[SPIFFS] upload : /var/folders/ky/fpk2bmgn4b30zjhyntzgtzhh0000gn/T/build3fadfb4cba027a61b568c284b4c3fb34.spiffs/CaptivePortal.spiffs.bin
[SPIFFS] reset : ck
[SPIFFS] port : /dev/cu.wchusbserial1410
[SPIFFS] speed : 115200
[SPIFFS] address: 0x100000

warning: espcomm_sync failed
error: espcomm_open failed
SPIFFS Upload failed!

[ivankravets]

Could you try other LD scripts? Like esp8266.flash.4m1m.ld http://docs.platformio.org/en/latest/platforms/espressif.html#id17

I recommend to ask help here https://gitter.im/esp8266/Arduino

[igrr]

Probably 3M flash erase taking longer than 10s timeout. This is really an
esptool bug.
If you select smaller FS (like 4m1), this will probably work.

On Wed, Dec 30, 2015, 09:20 Ivan Kravets [email protected] wrote:

Could you try other LD scripts? Like esp8266.flash.4m1m.ld
http://docs.platformio.org/en/latest/platforms/espressif.html#id17

I recommend to ask help here https://gitter.im/esp8266/Arduino

—
Reply to this email directly or view it on GitHub
https://github.com/platformio/platformio/issues/382#issuecomment-167914008
.

[itsjustvenky]

@ivankravets I just tested the SPIFFS upload with ESP-01 (1m-512FS) and ESP-12E (1m 3M FS) and it works without any issue. The 3M took little time ([SUCCESS] Took 316.21 seconds) for a payload

"/home/venkat/.platformio/packages/tool-mkspiffs/mkspiffs" -c data -p 256 -b 8192 -s 3125248 .pioenvs/esp12e/spiffs_image.bin

This is tested with PlatformIO, version 2.7.0.dev7

Right now I have tested only UART method. I will get back to soon on the remote FS upload.

thanks

[ivankravets]

@itsjustvenky you can't upload SPIFFS image over OTA. It's impossible.

[itsjustvenky]

@ivankravets Its possible

python espota.py -i <ESP_IP_address> -p <ESP_port> [-a password] -s -f <spiffs.bin>

Ref :

https://github.com/esp8266/Arduino/blob/master/tools/espota.py

[itsjustvenky]

@igrr Can you please shed some light on the OTA SPIFFS. thank you.

[ivankravets]

@itsjustvenky have you tried it? I don't know how does work OTA in ESP8266. From the official docs espota writes data to temporary region of the flash. How you are going to upload 3M of data when this region is small?

What is more, espota doesn't allow to specify direct address where data should be saved. The conclusion, OTA is ready only for firmware uploading.

[itsjustvenky]

@ivankravets Yes I just tried it

espota.py -i 192.168.0.105 -s -f  spiffs_image.bin 
Uploading........................................................................................................................................................................................................................................................................................................................................................................

and I can confirm the SPIFFS image is uploaded correctly. I can clearly the contents which I have uploaded in the FS.

The OTA firmware and SPIFFS work with the same OTA-mDNS-SPIFFS.ino code.Basically both are part of this class "ArduinoOTA".

I think the "spiffs.bin" already has the correct address and it will be taken by ArduinoOTA code.

[ivankravets]

@igrr that is true as for SPIFFS over OTA?

@Itsjustvenky which size of SPIFFS image have you just uploaded?

[itsjustvenky]

Here is the one with 3M SPIFFS image uploaded to ESP-12E (1m-3m-fs)

-rw-rw-r-- 1 venkat venkat 3125248 Dec 30 11:48 spiffs_image.bin
drwxrwxr-x 2 venkat venkat    4096 Dec 29 08:22 src
venkat@venkat-VirtualBox:~/Projects/platformio/test$ /home/venkat/Projects/back/platformio/platformio-usr-back/packages/framework-arduinoespressif/tools/espota.py -i 192.168.0.112 -s -f  spiffs_image.bin 
Uploading.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

[itsjustvenky]

here is the output from esp-12e

BEFORE UPLOAD (see the FS files list)

Chip ID: 0x126BA3
Hostname: ESP8266-OTA-126ba3
---- [ list fs files] ----------------
/esptool2-make1702
/hs_err_pid2471.log131834
/platformio.ini497
/Test.cpp102
------------------------------------
Failed to open cl_conf.txt.
No WiFi connection information available.
WiFi config changed.
new SSID: Venky
Wait for WiFi connection.
......
IP address: 192.168.0.112

 ets Jan  8 2013,rst cause:2, boot mode:(3,3)

load 0x4010f000, len 1264, room 16 
tail 0
chksum 0x42
csum 0x42
~ld

AFTER OTA SPIFFS UPLOAD

Chip ID: 0x126BA3
Hostname: ESP8266-OTA-126ba3
---- [ list fs files] ----------------
/espressif.json2183
/venkateshrao1702
/sindhu.log131834
/platformio.ini497
/Test.cpp102
------------------------------------
Failed to open cl_conf.txt.
No WiFi connection information available.
WiFi config changed.
new SSID: Venky
Wait for WiFi connection.
......
IP address: 192.168.0.112

[itsjustvenky]

@ivankravets I don't think the spiffs.bin image is loaded into temporary region. I am 100% sure that the image is directly copied into correct FS region directly OTA.

[igrr]

That's true, OTA upload overwrites SPIFFS directly. That's not very good
from reliability point of view, but that is the only solution for most
flash/fs size combinations.

On Wed, Dec 30, 2015, 14:25 Venkatesh [email protected] wrote:

@ivankravets https://github.com/ivankravets I don't think the
spiffs.bin image is loaded into temporary region. I am 100% sure that the
image is directly copied into correct FS region directly OTA.

—
Reply to this email directly or view it on GitHub
https://github.com/platformio/platformio/issues/382#issuecomment-167944780
.

[itsjustvenky]

@igrr thank you. So ArduinoOTA knows where to write the spiffs.bin right (I mean to FS memory location address) ?

[georgkreimer]

@ivankravets I tried 1M SPIFFS and that worked. (See log below)

Any idea how ESPlorer does their uploads? I know they don't upload a binary but I remember the uploads their were stable even for big files compared to other tools.

[SPIFFS] data : /Users/georgkreimer/Library/Arduino15/packages/esp8266/hardware/esp8266/2.0.0-rc2/libraries/DNSServer/examples/CaptivePortal/data
[SPIFFS] size : 1004
[SPIFFS] page : 256
[SPIFFS] block : 8192
/index.html
[SPIFFS] upload : /var/folders/ky/fpk2bmgn4b30zjhyntzgtzhh0000gn/T/build3fadfb4cba027a61b568c284b4c3fb34.spiffs/CaptivePortal.spiffs.bin
[SPIFFS] reset : ck
[SPIFFS] port : /dev/cu.wchusbserial1410
[SPIFFS] speed : 115200
[SPIFFS] address: 0x300000

Uploading 1028096 bytes from /var/folders/ky/fpk2bmgn4b30zjhyntzgtzhh0000gn/T/build3fadfb4cba027a61b568c284b4c3fb34.spiffs/CaptivePortal.spiffs.bin to flash at 0x00300000
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

[ivankravets]

Thanks for the reports. How about to avoid multiple new targets and implement new target named mkspiffs and then use it in pair with upload/uploadota? I mean that instead of upload, uploadota, uploadfs, uploadfsota make the next:

• mkspiffs - make SPIFFS image
• upload - if mkspiffs is specified then upload it, otherwise build firmware from the source code (generic way) and upload it
• uploadota - the same as upload, but will use OTA.

User guide:

• Upload firmware via USB = platformio run -t upload
• Upload firmware via OTA = platformio run -t uploadota
• Upload SPIFFS image via USB = platformio run -t mkspiffs -t upload
• Upload SPIFFS image via OTA = platformio run -t mkspiffs -t uploadota

WDYT?

[itsjustvenky]

@georgkreimer ESPlorer basically talks to nodemcu firmware on ESP and it uses the same SPIFFS principle internally. When flashing nodemcu firmware, we set the size of the flash and hence it will know how much free FS is available before upload.

[itsjustvenky]

@ivankravets The 4 option's are just perfect. For OTA "--upload-port" is mandatory. thanks.

[ivankravets]

@itsjustvenky platformio run -t mkspiffs -t upload looks so long? :)

[itsjustvenky]

Note : When uploading sketch or SPIFFS image via UART, the ESP should be in firmware mode (GPIO0 --> Gnd) , this is not needed incase of OTA (sketch as well as SPIFFS), just need to be in normal mode.

[itsjustvenky]

@ivankravets I am fine with it.

Actually we really don't need just making "spiffs" image, unless somebody wants to check the size of the spiffs image size before upload.

If we make the spiffs image, then it should be uploaded to ESP via UART or OTA.

So its upto you. thanks.

[georgkreimer]

Just as a thought... wouldn't OTA upload just mean that you use a different port than USB. So maybe you could have a wlan 'port' and keep upload and uploadfs as targets.

[ivankravets]

Guys, I decided to significantly simplify work PlatformIO. There are will no be here 4 or 6 new targets. I've added only the one new target uploadfs. The rest PlatformIO will handle on-the-fly. If you don't specify upload_port/--upload-port, then will be used uploading using Serial port, otherwise will be used OTA. For example,

• platformio run -t upload Upload firmware
• platformio run -t upload --upload-port 192.168.0.255 Upload firmware via OTA
• platformio run -t uploadfs Make SPIFFS image and upload it using Serial port
• platformio run -t uploadfs --upload-port 192.168.0.255 Make SPIFFS image and upload it using OTA

Please test it in the latest PlatformIO 2.7.0.dev10 http://docs.platformio.org/en/latest/installation.html#d-development-version

If it works properly, I will release 2.7.0 today. The #382 is the blocking issue for 2.7.0 milestone.

[ivankravets]

All detailed information is located in documentation http://docs.platformio.org/en/latest/platforms/espressif.html#uploading-files-to-file-system-spiffs

[itsjustvenky]

@ivankravets thanks. I have a question here.

The same --upload-port can be used to set the other serial port (if we have more than one) too right ? so then do you code differentiate between ip-address/URL address from UART port ?

We rarely use the "ip-address" notation to update a ESP via OTA, instead we use mDNS.
For example, a ESP can be address with name http://esp8266-A12343.local

[ivankravets]

@itsjustvenky in current scheme only IP address is allowed. If upload port is valid IP address then PlatformIO uses OTA.

[ivankravets]

@itsjustvenky Do you have any idea how to find difference between COM1 (real serial port) and COM2, where COM2(is dynamic name) is parked in hosts to 192.168.0.255?

[itsjustvenky]

@ivankravets if you are talking about mapping COM ports to "IP Address", then its not possible (afaik).

So it is possible for you add code to identify addresses that start with "http://" are also directed to OTA.

[ivankravets]

@itsjustvenky No here http.This is protocol. I've just implemented mDNS.local in PlatformIO 2.7.0.dev11. Please try it.

Guys, is it finish with this issue? Because we can improve it 1 year.

[itsjustvenky]

@ivankravets that's all I need. Thank you. Lets see other's opinion on this.

Happy New Year!!!

[ivankravets]

@itsjustvenky Thanks, you too! :christmas_tree: Don't forget to re-test that mDNS :)

[itsjustvenky]

Sure.. I will update you with results.

[0x2b3bfa0]

@ivankravets @itsjustvenky: I think that this issue is already prepared to be closed. I think that the SPIFFS is done and the OTA system was greatly improved. This issue served its purpose (and more). I :+1: to close this.

@ivankravets: :christmas_tree: Thank you very much! PlatformIO is building up very fast. I'll try to know how your code works to try to help you with a few PRs :smile:

[ivankravets]

Guys, all right. I mean that need to write all requests in the one post because is hard for me to implement new feature with each new comment :blush: More than 50 comments in this issue. Looks like "forum".

Need to start work on PlatformIO 3.0 https://github.com/platformio/platformio/milestones/3.0.0

I'll try to know how your code works to try to help you with a few PRs

I hope that I can resolve all issues/requests as for PlatformIO Core. I need help with adding new platforms, boards, etc. Here need to know how to use http://www.scons.org. Here are existing platforms/frameworks https://github.com/platformio/platformio/tree/develop/platformio/builder/scripts

Also, take a look at http://docs.platformio.org/en/latest/platforms/creating_platform.html

[itsjustvenky]

@ivankravets I have tested these following function's and all are working well as expected.

1) -t upload
2) -t uploadfs
3) -t upload --upload-port 192.168.0.105
4) -t uploadfs --upload-port 192.168.0.105
5) -t upload --upload-port venkatesp.local
6) -t uploadfs --upload-port venkatesp.local

We can close this now.

[ivankravets]

@itsjustvenky Thanks a lot! :+1:

Which is type of your OS? Could someone test it on Windows? I've just received new issue https://github.com/platformio/platformio/issues/417.

[itsjustvenky]

@ivankravets I have tested this on Ubuntu 14.04.3 LTS Desktop.

[georgkreimer]

@ivankravets Mac OS X El Capitan is mostly working except that known 3M SPIFFS issue. Unfortunately OTA has the same issue.

_first run for 1M SPIFFS_

Code/test ▶ platformio run --upload-port 192.168.4.1 -t uploadfs

[Wed Dec 30 16:36:36 2015] Processing huzzah (platform: espressif, board: huzzah, framework: arduino, build_flags: -Wl,-Tesp8266.flash.4m1m.ld)

"/Users/georgkreimer/.platformio/packages/tool-mkspiffs/mkspiffs" -c data -p 256 -b 8192 -s 1028096 .pioenvs/huzzah/spiffs.bin
/cl_conf.txt
(["uploadfs"], [".pioenvs/huzzah/spiffs.bin"])
"/Users/georgkreimer/.platformio/packages/framework-arduinoespressif/tools/espota.py" --debug --progress -i 192.168.4.1 -s -f .pioenvs/huzzah/spiffs.bin
16:36:37 [DEBUG]: Options: {'esp_ip': '192.168.4.1', 'image': '.pioenvs/huzzah/spiffs.bin', 'auth': '', 'esp_port': 8266, 'spiffs': True, 'debug': True, 'progress': True}
16:36:37 [INFO]: Starting on 0.0.0.0:24859
16:36:37 [INFO]: Upload size: 1028096
16:36:37 [INFO]: Sending invitation to: 192.168.4.1
16:36:37 [INFO]: Waiting for device...
Uploading: [============================================================] 100% Done...

16:37:01 [INFO]: Waiting for result...
16:37:01 [INFO]: Result: OK
============================================================================= [SUCCESS] Took 24.56 seconds =============================================================================

_second run for 3M SPIFFS_

Code/test ▶ platformio run --upload-port 192.168.4.1 -t uploadfs

[Wed Dec 30 16:37:20 2015] Processing huzzah (platform: espressif, board: huzzah, framework: arduino)

"/Users/georgkreimer/.platformio/packages/tool-mkspiffs/mkspiffs" -c data -p 256 -b 8192 -s 3125248 .pioenvs/huzzah/spiffs.bin
/cl_conf.txt
(["uploadfs"], [".pioenvs/huzzah/spiffs.bin"])
"/Users/georgkreimer/.platformio/packages/framework-arduinoespressif/tools/espota.py" --debug --progress -i 192.168.4.1 -s -f .pioenvs/huzzah/spiffs.bin
16:37:21 [DEBUG]: Options: {'esp_ip': '192.168.4.1', 'image': '.pioenvs/huzzah/spiffs.bin', 'auth': '', 'esp_port': 8266, 'spiffs': True, 'debug': True, 'progress': True}
16:37:21 [INFO]: Starting on 0.0.0.0:35555
16:37:21 [INFO]: Upload size: 3125248
16:37:21 [INFO]: Sending invitation to: 192.168.4.1
16:37:21 [INFO]: Waiting for device...
Uploading: [==================== ] 34%
16:37:53 [ERROR]: Error Uploading
scons: *** [uploadfs] Error 1
============================================================================= [ ERROR ] Took 33.11 seconds =============================================================================

[ivankravets]

Maybe set esp8266.flash.4m1m.ld as default LD script? Currently by default is esp8266.flash.4m.ld

@itsjustvenky do you have the same problems with uploading SPIFFS+OTA for 3m?

[itsjustvenky]

@ivankravets No

"/home/venkat/.platformio/packages/tool-mkspiffs/mkspiffs" -c data -p 256 -b 8192 -s 3125248 .pioenvs/esp12e/spiffs_image.bin

[ivankravets]

@itsjustvenky which board did you test? Looks like problem is only with board: huzzah?

[itsjustvenky]

ESP-12E basic board (not nodemcu)

like this one

http://i.ebayimg.com/00/s/MTYwMFgxNjAw/z/NL8AAOSwMmBViphp/$_1.JPG

[georgkreimer]

@itsjustvenky AP. I used the OTA-mDNS-SPIFFS.ino example. Using a huzzah module and this for UART All I had taken with me on vacation. I can test with other setups in like 3 days.

[itsjustvenky]

@georgkreimer can you paste the output from esp console (connect to UART) when doing OTA.
If there is reset then we can find what caused the issue.

[georgkreimer]

@itsjustvenky Sure.

�BBH�s���� ��������V������d$��|�l�<�l�c|����;�c�c��gn�dog���c8��lrl;lx�o��$��#g�|d����#��og�l��l �ng$'{���g��l s��'��d�n�

Chip ID: 0xA6046D
Hostname: ESP8266-OTA-a6046d
Wait for WiFi connection.
....................
Can not connect to WiFi station. Go into AP mode.
IP address: 192.168.4.1

See here. Left side platformIO, right side screen on UART

[itsjustvenky]

@georgkreimer There are chances that you might not have 4M flash on your ESP-12 (huzzah)

[georgkreimer]

@itsjustvenky You are probably right. Would make sense since it dies at 34%. I will check how much it really has and report back.

[itsjustvenky]

you can check using this

ESP.getFlashChipSize()

[georgkreimer]

;ld��|�$�|�$�c|ǃ��{�c�c��og�l'o���#x��l{l{$x�o��d��cg�|lČ��c��'o�$��l�'odgs���o��$r��g��l�o�

Chip ID: 0xA6046D
Hostname: ESP8266-OTA-a6046d
FlashChipSize: 4194304

[itsjustvenky]

I should try the OTA in AP mode.

[itsjustvenky]

@georgkreimer see this : https://github.com/esp8266/Arduino/issues/785

[georgkreimer]

Tried ESP.getFlashChipRealSize() but after reboot serial output is just gibberish.

I have another huzzah at home. Will test it when I'm back. Maybe I should open a separate issue when I can confirm it affects huzzah modules. So let's close this here.

[FWeinb]

It seams like data_dir isn't working correctly. Creating the spiffs.bin is working but when trying to upload the file I get:

"/Users/FWeinb/.platformio/packages/tool-mkspiffs/mkspiffs" -c data -p 256 -b 8192 -s 3125248 .pioenvs/nodemcu/spiffs.bin
warning: can't read source directory
scons: *** [.pioenvs/nodemcu/spiffs.bin] Error 1

Using the default folder is working.

[ivankravets]

@FWeinb Could you open separate issue with it? Thanks!

[ivankravets]

@FWeinb did you place data_dir to [platformio] section in platformio.ini?

[FWeinb]

Sorry for the delay. Final exams...
No i didn't know I have to place them there. Sorry for the trouble.

[msurguy]

So, just to be clear, this is the option that needs to be used in order for platformio run --target uploadfs command to work:

[platformio]
data_dir = data_dir

[ivankravets]

@msurguy you can omit this option if the directory is named as data.

[msurguy]

@ivankravets great, thanks for explaining, I was not able to find that in the docs

[ivankravets]

@msurguy we have documented this option in a few places:

• http://docs.platformio.org/en/latest/projectconf.html#data-dir
• http://docs.platformio.org/en/latest/platforms/espressif.html#uploading-files-to-file-system-spiffs

Did you see these pages? Maybe, need to rephrase what do we mean under data_dir?

[msurguy]

@ivankravets exactly, it is not clear from http://docs.platformio.org/en/latest/platforms/espressif.html#uploading-files-to-file-system-spiffs that data_dir in fact should be named as "data"

[ivankravets]

@msurguy thanks! I've just improved doc https://github.com/platformio/platformio/commit/9c15accbbb1e94be747c8cf7dfde307d340ade1f

Is it better now?

[msurguy]

@ivankravets beautiful! This does it!

[dalbert2]

Sorry if I'm missing something, but the new 2.0.x releases appear to break the upload_port setting in platformio.ini: it looks like it now interprets the upload_port as an IP address instead of a COM port (at least on windows platforms), so for example:

[env:esp12e]
platform = espressif8266
board = esp12e
framework = arduino
upload_port=COM5

Now results in:

Uploading .pioenvs\esp12e\firmware.bin
12:08:00 [DEBUG]: Options: {'timeout': 10, 'esp_ip': 'COM5', 'host_port': 46511, 'image': '.pioenvs\\esp12e\\firmware.bin', 'host_ip': '0.0.0.0', 'auth': '', 'esp_port': 8266, 'spiffs': False, 'debug': True, 'progress': True}
12:08:00 [INFO]: Starting on 0.0.0.0:46511
12:08:00 [INFO]: Upload size: 262160
Sending invitation to COM5 ..........
12:09:41 [ERROR]: No response from the ESP

Any ideas? I didn't see anything helpful in the release notes. How do I tell it to use a specific serial port for upload? (Note: if no upload port is specified, it auto-detects the first serial port, but that's not the one I need to use).

Also suggestion: when creating a new Arduino ESP8266 project, it would be very helpful for new users to include a comment in the auto-created platformio.ini referencing this page:
https://docs.platformio.org/en/latest/platforms/espressif8266.html#configuration

[ivankravets]

Do you have COM5 hostname?

[ivankravets]

@dalbert2 see https://github.com/platformio/platform-espressif8266/issues/134

Please open PlatformIO IDE > PIO Home > Platforms > Updates.