Libretime: silan < 0.3.3 tracks are cut in length during the import process (armhf)

[aresstokrat]

Alpha1 + debian jessie

After uploading the track and its being processed i found out the track cut from 3:23min to 1:48min in the library.
It happened to few tracks not all of them.

How can i find why its happened and fix this?

[aresstokrat]

seems like the problem is in "silan" package which is in last version installed "0.3.2-2+b2" ( problem is mentioned here https://forum.sourcefabric.org/discussion/17377/airtime-sometimes-cuts-tracks-in-half/p1 )

found fix packages from http://apt.sourcefabric.org/pool/main/s/silan/
but there is no package for jessie 'arm' architecture, and i dont know what to fix excatly and how

i think i should remove the silan package to stop it from work? or it will couse fatal errors?

[ghost]

There is a fix that might work for you. It works with Ubuntu Trusty (=Jessie) and is in the Sourcefabric repository. Maybe you can get it working, JohnnyC has it working on several ARM boxes (not PIs).
@hairmare did not like this solution, but I am also using it and it works perfectly here. See: #197
I think it compiles from APT sources.

[aresstokrat]

folowing https://github.com/LibreTime/libretime/issues/177#issuecomment-299195796
there is no armh package here:
http://download.opensuse.org/repositories/home:/hairmare:/silan/Debian_8.0/

trying to build from source http://download.opensuse.org/repositories/home:/hairmare:/silan/Debian_8.0/silan_0.3.3+nmu1.orig.tar.gz

failed, no configure

im not pro linux user, sorry

tryed to delete silan at all - stream jammed
installed again, deleted all media files from library, turned in and out fade to zero, imported file that was cut one more time - success, there is no cut.

[Robbt]

The other solution is to remove Silan. It is not essential according to other users who have tested this.
Just type sudo apt-get remove silan and reboot.

Did you try this and it didn't work ?
The Silan package is a major concern moving forward and the true proper fix requires interfacing directly with the Debian project to get the bugfix update.

[hairmare]

The opensuse build service doesn't have a runtime for building Debian on armh at the moment so I can't build silan for that arch 😞

There is an nmu for silan in the debian bugtracker but, since silan is unmaintained it has not seen any action (it probably also has it's own issues from Debians pov). There is an upstream issue for Debian Strech also tracking this. The strech fix there also fixes the wrong cutoffs on Jessie (as also reported in https://github.com/x42/silan/issues/3)

What model of raspberry pi are you on? I'll try to keep it in mind when I try to find a build host for arm nmu packages (and libretime packages)

[aresstokrat]

@Robbt Yes i removed with apt-get remove, and all streams jammed. I think silan is also responsible for cue points. Thats why removing it coused streams to get down immediately. And installing it back helped me to get streams back.
'in and out fade to zero' in settings of admin panel helped me to fix the bug (with silan installed).

@hairmare Im using "Raspberry Pi 3 Model B".

[ghost]

If you need a quick fix, I can only refer you again to my previous answer with Airtime's alternative solution, above. We have it working right now on ARM - not Rasberry Pi but re-imaged media centers (coffee shop radio stations) and AMD boxes - Deprogrammed Radio, Radio Rock UK and others.

[aresstokrat]

@Robby262 you mean link i paste in https://github.com/LibreTime/libretime/issues/214#issuecomment-305485231
?

it does't have armh architecture package for Jessie.
If you can tell how to integrate it, would be very helpful.

[ghost]

JohnnyC (team leader here) did it on Ubuntu Trusty, so it should be possible on Jessie, but he is in and out of hospital like a yoyo at the moment (this is also why that other forum has paused for a while). Maybe he built it from 3.1.1 source. The public version is behind the GIT 3.1.1 - a commit just three days after the public release fixed it, but they did not update the public release. If you can compile the Git 3.1.1 it does work.

[aresstokrat]

@Robby262

failed to build from git clone of https://github.com/x42/silan
will wait for johnnyC, wish him good health.

silan $ ./build-deb.sh

gbp:info: silan_0.3.3.orig.tar.gz does not exist, creating from 'master'
gbp:info: Exporting 'HEAD' to '/tmp/silan-0.3.3-0.1/silan-tmp'
gbp:info: Moving '/tmp/silan-0.3.3-0.1/silan-tmp' to '/tmp/silan-0.3.3-0.1/silan-0.3.3'
 dpkg-buildpackage -rfakeroot -D -us -uc -i -I
dpkg-buildpackage: пакет исходного кода silan
dpkg-buildpackage: версия исходного кода 0.3.3-0.1
dpkg-buildpackage: дистрибутив исходного кода unstable
dpkg-buildpackage: исходный код изменён Robin Gareus <[email protected]>
 dpkg-source -i -I --before-build silan-0.3.3
dpkg-buildpackage: архитектура узла armhf
dpkg-checkbuilddeps: Неудовлетворённые сборочные зависимости: libavformat-dev libavcodec-dev libavutil-dev libsndfile1-dev
dpkg-buildpackage: предупреждение: неудовлетворительные зависимости/конфликты при сборке; прерываемся
dpkg-buildpackage: предупреждение: (Используйте параметр -d, чтобы продолжить сборку.)
debuild: fatal error at line 1376:
dpkg-buildpackage -rfakeroot -D -us -uc -i -I failed
gbp:error: 'debuild -i -I -rfakeroot' failed: it exited with 29

[hairmare]

The build-dep.sh script and the rest of the the packaging efforts in the silan repo are rather outdated and haven't worked for a while since the packaging effort were in debian's repos for the last packages. I'll try to write up a short doc on how to build on a raspi over the weekend.

The Raspberry PI 3 is an ARMv8 architecture (armv8l if I'm not mistaken I was it's armv7l, maybe called arm64 on debian?). Im not sure what kinds of arm we should support. v7l for the RPi3 is certainly needed, probably also v6? for the RPi2. Maybe other depending on the CPUs in Robbys media-centers. I'd assume them to run an Allwinner CPU of some kind, at least all my cheap arm boxes from China have been using those for ages now (mostly armv7 compatible with softfloat).

The sourcefabric packages should work just fine. As a project I don't think it would be legit to add them to the installer. The package mirror on apt.sourcefabric.org is not ours for taking and it's unclear what patches where applied to the package (I couldn't find the source deb package). While this violates the 4 freedoms from the Free Software Definition it's probably not that much of an issue for end users. edit: I later found the source package and I didn't wan't to come across so harsh either. Fact is I can't .deb very well yet :P

[ghost]

I already covered this, but you don't believe it. The silan licensing extremely clearly defined on every single page of the source and in the docs. The guy who looks after the packaging of LiquidSoap and Silan for SourceFabric was, and probably is, Daniel James who is/was one of the LiquidSoap devs.
Maybe, if we asked nicely, he would do that part for us too. He is a very helpful and nice guy.

[hairmare]

I built an armhf package locally an a dockcross/linux-armv7 container that might work. I was not able to test it since I don't own a Raspberry Pi 3 board.

I had to wrap it into a tarball so I could upload it here: silan_0.3.3~nmu1_armhf.deb.tar.gz

Workaround

To install the local nmu build i uploaded to github you can run the following commands:

curl -L -O https://github.com/LibreTime/libretime/files/1049738/silan_0.3.3.nmu1_armhf.deb.tar.gz
tar xvf silan_0.3.3.nmu1_armhf.deb.tar.gz
sudo dpkg -i silan_0.3.3~nmu1_armhf.deb

Please let us know if the package works. If it does I'll try and find a more permanent home for it.

Build process

If anyone is interested, this is how I built the package:

# install packaging tools
sudo apt-get install -y build-essential devscripts lintian diffutils patch patchutils

# add source repos that contain build-deps
sudo tee /etc/apt/sources.list <<EOD
#------------------------------------------------------------------------------#
#                   OFFICIAL DEBIAN REPOS                    
#------------------------------------------------------------------------------#

###### Debian Main Repos
deb http://deb.debian.org/debian/ stable main contrib non-free
deb-src http://deb.debian.org/debian/ stable main contrib non-free

deb http://deb.debian.org/debian/ stable-updates main contrib non-free
deb-src http://deb.debian.org/debian/ stable-updates main contrib non-free

deb http://deb.debian.org/debian-security stable/updates main
deb-src http://deb.debian.org/debian-security stable/updates main

deb http://ftp.debian.org/debian jessie-backports main
deb-src http://ftp.debian.org/debian jessie-backports main

#------------------------------------------------------------------------------#
#                   ADDITIONAL REPOS
#------------------------------------------------------------------------------#

###### Emdebian ARM Tools Repos
deb http://emdebian.org/tools/debian/ jessie main
deb-src http://emdebian.org/tools/debian/ jessie main
EOD
sudo apt-get update

# install silan build deps
sudo apt-get build-dep -y -aarmhf silan
# add last missing dep
sudo apt-get install -y libsndfile1-dev:armhf

# grab debian silan tree
git clone git://anonscm.debian.org/pkg-multimedia/silan.git debian-silan
cd debian-silan

# apply NMU patches one by one
curl 'https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=855319;filename=0001-Bump-to-0.3.3-from-upstream.patch;msg=5' | git apply
curl 'https://bugs.debian.org/cgi-bin/bugreport.cgi?att=2;bug=855319;filename=0002-debian-watch-Update-from-deprecated-githubredir-serv.patch;msg=5' | git apply
curl 'https://bugs.debian.org/cgi-bin/bugreport.cgi?att=3;bug=855319;filename=0003-debian-patches-Removed-due-to-being-in-upstream.patch;msg=5' | git apply
curl 'https://bugs.debian.org/cgi-bin/bugreport.cgi?att=4;bug=855319;filename=0004-debian-control-bump-to-Standards-Version-3.9.6.patch;msg=5' | git apply

# overwrite the source code with latest release
curl -L -O https://github.com/x42/silan/archive/v0.3.3.tar.gz
tar --strip-components=1 -xvf v0.3.3.tar.gz

# tag local version as 0.3.3~nmu
dch --local ~nmu

# build the actual package
CONFIG_SITE=/etc/dpkg-cross/cross-config.armhf \
DEB_BUILD_OPTIONS=nocheck \
PKG_CONFIG_PATH=/usr/lib/arm-linux-gnueabihf/pkgconfig/ \
dpkg-buildpackage -aarmhf -b -uc -us

# install it to check if it installs
sudo dpkg -i ../silan_0.3.3~nmu1_armhf.deb

[hairmare]

I made it clear that this issue is about arm in the title. There is also https://github.com/LibreTime/libretime/issues/177#issuecomment-299195796 with an apt based workaround for i586 and x86_64 archs.

[aresstokrat]

@hairmare GREAT news! Thank you!
removed prev. silan and installed your package.
Installed without errors.

It coused 'neverending import bug'. Track after upload in status "import pending' for hours.

What log can we dig to know whats happening?

[hairmare]

First of all try running silan on a standalone file. Does that show any errors?

Next you can stop the service and run analyzer manually to check for errors:

systemctl stop airtime_analyzer
sudo -u www-data airtime_analyzer

Wait for Listening for messages... and then upload a file to see what happens.

If there still is no reasonable error output then you can add the --debug argument to airtime_analyzer which will output heaps of info.

[aresstokrat]

What i did:

1. run 'silan' standalone - output normal, working, version 0.3.3. .
2. stoped service analyzer
3. sudo -u www-data airtime_analyzer

And upload gone completly FINE !!! More than that 2 tracks that was marked as "import failed" (after 2 hours of pending) immidiiatly marked as 'successfuly imported'. I tryed refresh page, its not browsers cache.

Dont know what to say. ;-( How could manual run of service help that thing?

Bug is fixed, length of tracks that was cut before now is normal, win 'fade in fade out' points enabled!

[hairmare]

Good to hear that the binary is working on your Raspi3 🎉

How could manual run of service help that thing?

It shouldn't :( Maybe analyzer was hanging due to other reasons :/

Does it work when you switch back to running silan/analyzer as a service?

Also, you will have to re-import the files that have wrong info since that info is still in the database.

I added a hint about the armhf silan binaries to the draft release notes for 3.0.0-alpha.2.

[aresstokrat]

its working fine as a service! :+1:

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please chat to us on discourse or ask for help on our chat if you have any questions or need further assistance with this issue.