Ungoogled-chromium: Building on openSUSE (various questions)

Created on 5 Aug 2018  路  47Comments  路  Source: Eloston/ungoogled-chromium

I am interested to build ungoogled-chromium on systems which run:

  • openSUSE Leap 15.0 (64 bit)
  • openSUSE Tumbleweed (32 bit)

I have read through the BUILDING.md but I have some questions before stepping into it:

Configuring the build

Normally when I download the source code of some software there are these standard steps:

configure (with options)
make
make install

but obviously the build process of this software is different. What is the reason for this and how could one configure the build?

CFLAGS

Usually I like to set custom CFLAGS and CXXFLAGS like -O2 -pipe -march=native to optimize the build for the current system. How can I do this for ungoogled-chromium please?

Target directory

For good and clean separation I like to put custom build software which is not part of the official repos in /opt/<program-name>. In this case that would be /opt/ungoogled-chromium. How can I do that?

Existing chromium installation

Currently on Leap 15.0 I have openSUSE's chromium package installed. Do I need to uninstall it before building ungoogled-chromium?

System package updates and dependencies

I suppose ungoogled-chromium is not a 100% self contained program and depends on installed system libraries. Is there any danger that it may break after a zypper update? If yes - could you recommend steps to minimize the chance for this, so that one doesn't need to rebuild it (except for new versions).

Profiles

Will ungoogled-chromium detect my existing ~/.config/chromium or should I remove it and start from scratch?

Default settings

In non-ungoogled chromium there is this nightmare of first run which has all kinds of features enabled and loads some home page which the user may have never wanted to send one's IP address to, yet the developers have decided that appropriate. For improved security and privacy I would like to have "out of the box":

Disabled:

chrome://settings/content/cookies
chrome://settings/content/location
chrome://settings/content/camera
chrome://settings/content/microphone
chrome://settings/content/notifications
chrome://settings/content/javascript
chrome://settings/content/flash
chrome://settings/content/backgroundSync
chrome://settings/content/automaticDownloads
chrome://settings/content/unsandboxedPlugins
chrome://settings/content/midiDevices
chrome://settings/content/protectedContent
chrome://settings/languages - disable spell check and translation
chrome://settings/cloudPrinters

chrome://inspect/#devices
    - Disable "Discover USB devices" and "Discover network targets"

chrome://flags/#disable-hyperlink-auditing = disabled
chrome://flags/#shared-array-buffer = disabled
chrome://flags/#enable-site-per-process = enabled

chrome://settings/onStartup
    - Open a specific page or set of pages: about:blank

chrome://settings/privacy
    Disable:

    - Use a web service to help resolve navigation errors
    - Use a prediction service to help complete searches and URLs typed in the address bar
    - Use a prediction service to load pages more quickly
    - Automatically send some system information and page content to Google to help detect dangerous apps and sites
    - Protect you and your device from dangerous sites
    - Allow sites to check if you have payment methods saved
    - Continue running background apps when Chromium is closed

    Enable:

    - Send a "Do Not Track" request with your browsing traffic

chrome://settings/searchEngines
    - Set default search engine to nothing: https://%s

chrome://settings/passwords
    - Auto Sign-in = disabled

Currently I do this by using a blank template profile with all these settings which I copy to each user's ~/.config/chromium to make it a starting point for new users. But considering I will be building a customized software I wonder if it would be possible to have all these settings by default. Is that possible (how)?

question
Source
picture ghost

Most helpful comment

Friends, I made it! Thank you for your help :)

I have built LLVM from source and then used it to build ungoogled chromium (UGC). Here is the little script I wrote for the purpose, in case anyone is interested to use it:

#!/bin/bash

START_TIME=$SECONDS

workdir='/tmp/download'
svnurl='https://llvm.org/svn/llvm-project'
src='llvm-project'
build='llvm-build'
dest='/opt/llvm'

echo "鈻堚枅鈻堚枅 Remove ${workdir}/${src}"
rm -rf "${workdir}/${src}"
mkdir -p "${workdir}/${src}"

echo '鈻堚枅鈻堚枅 Checkout LLVM'
cd "${workdir}/${src}"
svn co -q "${svnurl}/llvm/trunk" llvm

echo '鈻堚枅鈻堚枅 Checkout Clang'
cd "${workdir}/${src}"
cd llvm/tools
svn co -q "${svnurl}/cfe/trunk" clang

echo '鈻堚枅鈻堚枅 Checkout Extra Clang Tools'
cd "${workdir}/${src}"
cd llvm/tools/clang/tools
svn co -q "${svnurl}/clang-tools-extra/trunk" extra

echo '鈻堚枅鈻堚枅 Checkout LLD linker'
cd "${workdir}/${src}"
cd llvm/tools
svn co -q "${svnurl}/lld/trunk" lld

echo '鈻堚枅鈻堚枅 Checkout Polly Loop Optimizer'
cd "${workdir}/${src}"
cd llvm/tools
svn co -q "${svnurl}/polly/trunk" polly

echo '鈻堚枅鈻堚枅 Checkout Compiler-RT (required to build the sanitizers)'
cd "${workdir}/${src}"
cd llvm/projects
svn co -q "${svnurl}/compiler-rt/trunk" compiler-rt

echo '鈻堚枅鈻堚枅 Checkout Libomp (required for OpenMP support)'
cd "${workdir}/${src}"
cd llvm/projects
svn co -q "${svnurl}/openmp/trunk" openmp

echo '鈻堚枅鈻堚枅 Checkout libcxx and libcxxabi'
cd "${workdir}/${src}"
cd llvm/projects
svn co -q "${svnurl}/libcxx/trunk" libcxx
svn co -q "${svnurl}/libcxxabi/trunk" libcxxabi

echo '鈻堚枅鈻堚枅 Get the Test Suite Source Code'
cd "${workdir}/${src}"
cd llvm/projects
svn co -q "${svnurl}/test-suite/trunk" test-suite

echo '鈻堚枅鈻堚枅 Configure and build LLVM and Clang'
cd "${workdir}"
rm -rf "${build}"
mkdir "${build}"
cd "${build}"

export CFLAGS="-O3 -pipe -march=native"
export CXXFLAGS="${CFLAGS}"

cmake -G Ninja \
    -DCMAKE_INSTALL_PREFIX="${dest}" \
    -DCMAKE_BUILD_TYPE=Release \
    "${workdir}/${src}/llvm"

if [[ $? -ne 0 ]]
then
    echo '鈻堚枅鈻堚枅 cmake exited with error'
    exit 1
fi

echo '鈻堚枅鈻堚枅 Running ninja'
if ! ninja -j$(nproc) -C "${workdir}/${build}"; then
    echo '鈻堚枅鈻堚枅 Ninja exited with error'
    exit 1
fi

ELAPSED_TIME=$(($SECONDS - $START_TIME))

hours=$((ELAPSED_TIME / 3600))
seconds=$((ELAPSED_TIME % 3600))
minutes=$((seconds / 60))
seconds=$((seconds % 60))

echo "鈻堚枅鈻堚枅 Elapsed time: $hours hour(s) $minutes minute(s) $seconds second(s)"

Note: Perhaps the -DCMAKE_INSTALL_PREFIX is not needed when ninja is used (and not make). Building LLVM takes about 1 hour here.

I also added the following line to the previous script I shared (the one I use to build UGC itself):

export PATH="/tmp/llvm-build/bin:$PATH"

before the line which runs build.sh.

_First impression:_ UGC feels faster than standard Chromium! I don't know if it is due to it being a custom build for the particular machine but it is great.

Some other things I would like to discuss with you if I may:

  • Along the lines of building a portable version: does /etc/default/chromium matter at all? If not - which is the "master" settings file where one can set default preferences for all users on the system?

  • Spell check doesn't seem to work. Is it because it needs connection to Google?

  • Earlier you mentioned that I need to modify the Inox patches in order to have default settings the way I want them. However looking at them I see they include specific line numbers which can change with any new version of Chrome, so it seems to need a continuous careful work which I am not sure I will have the time to do. Isn't there a simpler way to approach this?

  • chrome://net-internals/#hsts section HSTS/PKP shows a link https://www.ch40m1um.qjz9zk/hsts which made me wonder: Is it possible that due to the UGC's specific replacement of host names the "HSTS preloaded list" of Chrome doesn't work? (which has a certain security implication)

  • While browsing chrome://settings/ I was watching a ss -tuapn on the side and noticed a TCP connection to 104.20.23.46:443. At that time I was on the Language setting. Then I ran rcnetwork restart and tried to reproduce it but I could not. Some minutes later when I opened several chrome://flags tabs it showed up again. Since then I haven't seen it. I have no idea what this host is but considering its location it may be worth checking if it is an attempt of Chrome to connect to something:

[~]: geoiplookup 104.20.23.46
GeoIP Country Edition: US, United States
GeoIP City Edition, Rev 1: US, CA, California, San
  Francisco, 94107, 37.769699, -122.393303, 807, 415
  GeoIP ASNum Edition: AS13335 CloudFlare, Inc.

  • uBlock/uMatrix: Do you happen to know where "Export to cloud storage" actually uploads? When I click it I see no Internet connections but it still seems to work.

  • What benefit has using --set-ipv6-probe-false?

  • Does installing an extension as CRX result in possibility to have it auto-updated from Chrome store? I am just looking for a way to install from file "HTTPS Everywhere" but EFF's site seems to offer only CRX downloads.

Re. the build process:

As you can see in my UGC build script there are 3 sed lines. The first 2 of them insert the specific CFLAGS. The third one modifies the ninja line to use the actual number of cores instead of the default (10). My questions are:

  • Do I need to do the first 2 inserts in the build.sh itself or will it be enough to simply export the CFLAGS in my own bash script?

  • Do you think you could probably include my modification of the build.sh by adding -j$(nproc) to the ninja command? It seems a universal modification which would benefit everyone.

  • I have an old 32-bit laptop for which I am willing to build UGC too. But I am afraid it may be a terribly slow process (if possible at all because it has a small disk and not so much RAM). How can I use the stronger desktop machine to create a binary for the laptop? I wonder if it just a matter of different CFLAGS (and what should they be) or something else? I hope you can shed some light.

picture ghost on 23 Sep 2018
🎉1 👍1

All 47 comments

Let me attempt to answer your questions at once.

ungoogled-chromium is Google Chromium sans integration with Google. There are also some changes to improve privacy, control, and transparency, but they shouldn't break compatibility with regular Chromium. So expect all existing settings, executable names, and branding to be exactly as they are in regular Chromium.

For convenience, ungoogled-chromium first generates packaging scripts for various systems. On Linux, we try our best to match the Chromium packages for the respective distributions. They are otherwise standard packaging scripts; all the actual building steps occur within them. You are free to add, remove, and modify as you would with any other packaging script. This will become more apparent once I've finished the work I'm doing in the redesign branch.

Becuase of ungoogled-chromium's privacy focus, settings already default to enhance privacy in new profiles. This is pretty standard across all Chromium derivatives, and it is no different here. In ungoogled-chromium's case, most of the defaults come from Inox browser; you can find its source code patch in its patches directory.

Eloston picture Eloston on 8 Aug 2018

Thanks for the reply!

ungoogled-chromium is Google Chromium sans integration with Google. [...]

OK. This answers the Profiles question.

For convenience, ungoogled-chromium first generates packaging scripts for various systems. [...]

I have never packaged software so for the moment I am not concerned about building a distributable package. So that convenience is not of primary importance to me. As you may have guessed from the nature of the questions: I am rather interested in being able to configure/customize the build, install it where I want, git-update/recompile it when appropriate and maintain it this way for all computers I manage. And all this - whithout creating conflict with existing (distro-provided) packages, their configs or file paths. Do you think you could please answer the rest of the questions? (as I don't know python and finding the answers by looking at your code was difficult for me, even after a few hours)

[...] most of the defaults come from Inox browser [...]

When you say most that implies there are others which don't come from that browser. Could you please elaborate? Also - what should one do if one wants to modify Inox's (or the rest of the) settings?

ghost picture ghost on 9 Aug 2018

I am rather interested in being able to configure/customize the build, install it where I want, git-update/recompile it when appropriate and maintain it this way for all computers I manage. And all this - whithout creating conflict with existing (distro-provided) packages, their configs or file paths.

I see. Thanks for giving me the high-level objective.

In that case, you may be more interested in the Portable Linux build process, which are portable binaries. If you want to use the openSUSE configuration from ungoogled-chromium, you can choose to use the opensuse config bundle instead of linux_portable, but still use the linux_simple packaging scripts. Follow the instructions for Other Linux distributions and follow the packaging instructions for an Archive. Before you run build.sh, have a look through the comments inside and make changes as necessary.

When you say most that implies there are others which don't come from that browser. Could you please elaborate? Also - what should one do if one wants to modify Inox's (or the rest of the) settings?

There are many Chromium derivatives, i.e. independent free software projects spun off of Chromium's code. One of them is Inox, and ungoogled-chromium uses patches from it (as you may already know from the README). As with any free software project, the developers are free to use code from others or not. ungoogled-chromium happens to be a project that is a combination of borrowed changes and new changes.

If you are interested in modifying the default settings Inox changes, here are two straight-forward options you may be interested in:

  1. Modify Inox's patch before the build starts
  2. Apply the patches, modify the source code manually, then run the build commands.
Eloston picture Eloston on 9 Aug 2018

Configuring the build + CFLAGS

You can "configure" the build by choosing a flavor and then, if you want to change it further, manually changing its flags. To do what you asked, you could build the compressed tar archive (see the bottom of BUILDING.md) and edit the generated build.sh file. There you'll see at which line you should put the extra flags you want (it's commented).

For good and clean separation I like to put custom build software which is not part of the official repos in /opt/. In this case that would be /opt/ungoogled-chromium. How can I do that?

The static/portable/tar_archive build will generate a compressed folder, which you can extract to /opt/ungoogled-chromium or anywhere you want. You'd run it by simply opening the chromium binary inside it.

Currently on Leap 15.0 I have openSUSE's chromium package installed. Do I need to uninstall it before building ungoogled-chromium?

For the static/simple build, no you shouldn't need to uninstall it. If you want RPMs though, I believe you'd need to. Maybe someone more experienced with openSUSE will give a better answer (neither me or eloston use it).

Is there any danger that it may break after a zypper update? If yes - could you recommend steps to minimize the chance for this, so that one doesn't need to rebuild it (except for new versions).

Build the archive version and you shouldn't have any problems after updates.

Will ungoogled-chromium detect my existing ~/.config/chromium or should I remove it and start from scratch?

It should work with it normally. No need to remove it.

In non-ungoogled chromium there is this nightmare of first run which has all kinds of features enabled and loads some home page which the user may have never wanted to send one's IP address to

This simply won't ever happen in ungoogled-chromium. It won't connect to any domains on startup, unless you added an extension that explicitly does so (for example, uBlock Origin checking for list updates). All Google domains are replaced in the source code before building, so there's no chance of that happening, really.

The defaults are similar to the ones you posted, minus JS (it's enabled by default), search engine (DuckDuckGo by default) and cookies (1st-party enabled, 3rd-party blocked). I am not sure how you could change the defaults for a fresh build, but really wouldn't bother do. UGC defaults are more than sane enough privacy-wise.

9Morello picture 9Morello on 9 Aug 2018

Thank you for the explanations and sorry for the late reply.

I will look at the build process you guys described.

search engine (DuckDuckGo by default) [...] UGC defaults are more than sane enough privacy-wise.

DDG is a bad choice as it is hosted on Amazon:

[~]: host duckduckgo.com
duckduckgo.com has address 46.51.179.90
duckduckgo.com has address 176.34.155.23
duckduckgo.com has address 79.125.105.113
duckduckgo.com mail is handled by 20 in2-smtp.messagingengine.com.
duckduckgo.com mail is handled by 10 in1-smtp.messagingengine.com.
[~]: host 46.51.179.90
90.179.51.46.in-addr.arpa domain name pointer ec2-46-51-179-90.eu-west-1.compute.amazonaws.com.
[~]: host 176.34.155.23
23.155.34.176.in-addr.arpa domain name pointer ec2-176-34-155-23.eu-west-1.compute.amazonaws.com.
[~]: host 79.125.105.113
113.105.125.79.in-addr.arpa domain name pointer ec2-79-125-105-113.eu-west-1.compute.amazonaws.com.

So if that is supposed to be a privacy respecting program https://www.startpage.com (or something else) looks like a better choice. However I have no idea how to make startpage default search engine and use its POST search feature. Do you know about that?

ghost picture ghost on 29 Aug 2018

If you want to use the openSUSE configuration from ungoogled-chromium, you can choose to use the opensuse config bundle instead of linux_portable, but still use the linux_simple packaging scripts.

How do I do this please?

Follow the instructions for Other Linux distributions and follow the packaging instructions for an Archive. Before you run build.sh, have a look through the comments inside and make changes as necessary.

The link gives 404. Also the documentation for "Any Linux distribution" is referencing a file which doesn't exist (which I filed in a separate issue #473).

ghost picture ghost on 29 Aug 2018

@anchev

However I have no idea how to make startpage default search engine and use its POST search feature. Do you know about that?

See #105, which is the same thing but for DDG.

Eloston picture Eloston on 29 Aug 2018

@anchev

The link gives 404

I changed the section names some time after that comment. It's "Any Linux distribution" now.

Eloston picture Eloston on 29 Aug 2018

Thanks.

BTW another thing:

To do what you asked, you could build the compressed tar archive (see the bottom of BUILDING.md) and edit the generated build.sh file. There you'll see at which line you should put the extra flags you want (it's commented).

I can't find such path:

# See build/toolchain/linux/unbundle/ in the Chromium source for more details.

Am I missing something?

ghost picture ghost on 29 Aug 2018

@anchev

Am I missing something?

Same answer as https://github.com/Eloston/ungoogled-chromium/issues/473#issuecomment-417107870

Eloston picture Eloston on 29 Aug 2018

Thanks.

OK. I have been following closely the instructions for "Any Linux distribution" and the additional info given here. On openSUSE Leap 15.0 there is no LLVM 6.0 but there is 5, so that is what I installed (along with all the other packages listed in the zypper in command in the openSUSE section). BTW I also needed to install mozilla-nss-devel because I was getting an error that Package nss was not found.

So I added my 2 flag lines in build.sh on lines 33-34 as advised by @9Morello:

export CFLAGS="-O2 -pipe -march=native"
export CXXFLAGS="${CFLAGS}"

and I ran ./ungoogled_packaging/build.sh which ended with:

/tmp/download/ungoogled-chromium/build/src
+ export AR=llvm-ar
+ AR=llvm-ar
+ export NM=llvm-nm
+ NM=llvm-nm
+ export CC=clang
+ CC=clang
+ export CXX=clang++
+ CXX=clang++
+ export 'CFLAGS=-O2 -pipe -march=native'
+ CFLAGS='-O2 -pipe -march=native'
+ export 'CXXFLAGS=-O2 -pipe -march=native'
+ CXXFLAGS='-O2 -pipe -march=native'
+ ./tools/gn/bootstrap/bootstrap.py -o out/Default/gn -s
Building gn manually in a temporary directory for bootstrapping...
ninja: Entering directory `/tmp/tmpK68wZb'
[415/415] LINK gn
+ ./out/Default/gn gen out/Default --fail-on-unused-args
ERROR at //build/config/posix/BUILD.gn:56:28: Undefined identifier in string expansion.
      "CR_LIBCXX_REVISION=$libcxx_svn_revision",
                           ^------------------
"libcxx_svn_revision" is not currently in scope.
See //build/config/compiler/BUILD.gn:1150:18: which caused the file to be included.
    configs += [ "//build/config/posix:runtime_library" ]
                 ^-------------------------------------

How do I fix this please?

ghost picture ghost on 29 Aug 2018

How do I fix this please?

I am resolving this in #472

Eloston picture Eloston on 30 Aug 2018

Thanks for the info.
Could you please just confirm if that is due to LLVM version, i.e. if it is mandatory to have 6.0+?

ghost picture ghost on 30 Aug 2018

I don't think so, but I haven't investigated yet.

Eloston picture Eloston on 30 Aug 2018

Then I will have to wait till you fix the other issue. Thank you.

ghost picture ghost on 30 Aug 2018

Hi,

I see you closed #472 through a particular commit. But repeating the same build process now gives me a much longer list of problems. I share through susepaste.org as I am not sure if it would be OK to paste such long text here:

https://susepaste.org/c518a638

ghost picture ghost on 31 Aug 2018

What version of LLVM are you using?

Eloston picture Eloston on 31 Aug 2018 
[~]: rpm -q llvm
llvm-5.0.1-lp150.4.3.1.x86_64
ghost picture ghost on 31 Aug 2018

Please upgrade to at least LLVM 6. I'm not sure if a newer version is required for version 68.

Eloston picture Eloston on 31 Aug 2018

@anchev have a look at https://github.com/Eloston/ungoogled-chromium/issues/490
you can install llvm 6 by adding this repo 

zypper addrepo -f https://download.opensuse.org/repositories/devel:/tools:/compiler/openSUSE_Leap_42.3/ new-devels-clang6llvm

edit the link to match your opensuse version... also check if the repo exist for your opensuse version

intika picture intika on 11 Sep 2018

OK. Upgraded packages:

[~]: rpm -q llvm
llvm-6.0.1-lp150.530.2.x86_64
[~]: rpm -q clang
clang-6.0.1-lp150.530.2.x86_64
[~]: rpm -q lld
lld-6.0.1-lp150.530.2.x86_64

Unfortunately I am still getting errors:

https://susepaste.org/f455519c

ghost picture ghost on 14 Sep 2018

Thanks.

BTW I didn't get an email notification for that commit although it is in an issue opened by me. I wouldn't know about it if I didn't visit the web page here. Is that normal?

Also I am still getting errors (and warnings):

https://susepaste.org/8e917bc9

ghost picture ghost on 19 Sep 2018

A side note: I had to install also alsa-devel, libuuid-devel and libpulse-devel because otherwise I was getting other errors (as "Missing uuid.h" and similar). You may want to add these to the list of required packages in the docs.

ghost picture ghost on 19 Sep 2018

@anchev: Please see https://github.com/Eloston/ungoogled-chromium/issues/494#issuecomment-422631189, I believe intika faced the same issue and solved it by using a recent version of lld.

xsmile picture xsmile on 19 Sep 2018
👍1

I am already using the latest llvm and lld (6.0.1) for Leap 15.0 from devel:tools:compiler repo as listed here:

https://software.opensuse.org/package/lld

As far as I can see @intika is using Leap 42.3 and a repo of a specific user, which is neither listed on software.opensuse.org, nor even a community repo. Even if there was such repo for Leap 15.0 I would still be highly reluctant to use it. I was even hesitant about the one which I currently use as it is still an unofficial one.

I wonder how come this was supposed to work version 6 and now it suddenly requires something else?

ETA: I see that LLVM 7.0 was released today. Why would a software which was released before it require it for building itself?

ghost picture ghost on 19 Sep 2018

@anchev
The repo i am using is listed on opensuse.org >> https://software.opensuse.org/package/lld8
Here is the version for your distro https://build.opensuse.org/package/show/home%3Akhnazile%3Avideo/llvm

I am also skeptical when using non official repo but this beast is a pain in a.. to build so just using what ever i find to get it to build then once we know what it need exactly we can look for official releases OR just build llvm/clang/lld from sources it's not that big of a deal

intika picture intika on 19 Sep 2018

As far as I can see @intika is using Leap 42.3 and a repo of a specific user, which is neither listed on software.opensuse.org, nor even a community repo. Even if there was such repo for Leap 15.0 I would still be highly reluctant to use it. I was even hesitant about the one which I currently use as it is still an unofficial one.

Don't affirm something without being sure ! dig a little bit ! https://github.com/Eloston/ungoogled-chromium/issues/447#issuecomment-422868920

I wonder how come this was supposed to work version 6 and now it suddenly requires something else?

ETA: I see that LLVM 7.0 was released today. Why would a software which was released before it require it for building itself?

You can build chromium a million ways ... indeed v69 sources packages from various linux distros (opensuse included) use gcc-7 to build it and does not require clang at all (nor lld nor llvm)

intika picture intika on 19 Sep 2018

@anchev

ETA: I see that LLVM 7.0 was released today. Why would a software which was released before it require it for building itself?

Both you and @intika are using linux_portable to build, which enables newer optimizations like LTO, CFI, etc. Google likes to push the bleeding edge for new compiler optimizations, so they use their own build of LLVM of a specific revision for Chrome/Chromium builds (which is specified in tools/clang/scripts/update.py). This is typically a revision that is newer than any released version of LLVM.

In addition, it seems likely that linux_portable actually requires at least LLVM 8; see https://github.com/Eloston/ungoogled-chromium/issues/494#issuecomment-422861767

Independent of linux_portable, there is also the openSUSE packaging that uses GCC and doesn't use these newer optimizations, but no one has made the initiative to update it yet.

Eloston picture Eloston on 19 Sep 2018

using what ever i find to get it to build

I am not really a fan of such approach. It makes even less sense along the lines of the current project: one is trying to free oneself from untrusted stuff and the way to it is not to use whatever. (imo)

build llvm/clang/lld from sources it's not that big of a deal

That is what I would rather be interested to try. Currently looking for info how to do it.

there is also the openSUSE packaging

Yeah, but as we talked above - it seems not to give the flexibility I am looking for. If it was more customizable then perhaps that would be the right thing to use.

BTW the whole build process takes about 2 hours here. I wonder if there is a way to optimize it. Any suggestions? I am using a simple bash script for the whole thing:

#!/bin/bash

workdir='/tmp/download'
giturl='https://github.com/Eloston/ungoogled-chromium.git'
src='ungoogled-chromium'

echo "鈻堚枅鈻堚枅 Remove ${workdir}/${src}"
mkdir -p "${workdir}"
cd "${workdir}"
rm -rf "${src}"

echo '鈻堚枅鈻堚枅 Download from GIT'
git clone "${giturl}" "${src}"
cd "${src}"

mkdir -p build/src
./get_package.py linux_simple build/src/ungoogled_packaging
cd build/src

BUILD_SCRIPT="${workdir}/${src}/build/src/ungoogled_packaging/build.sh"

echo '鈻堚枅鈻堚枅 Adding custom build flags'
sed -i -n 'p;33a export CXXFLAGS="${CFLAGS}"' "${BUILD_SCRIPT}"
sed -i -n 'p;33a export CFLAGS="-O2 -pipe -march=native"' "${BUILD_SCRIPT}"

sed -i "s/ninja -C/ninja -j$(nproc) -C/g" "${BUILD_SCRIPT}"

cat "${BUILD_SCRIPT}"

while true; do
    read -n 1 -p "鈻堚枅鈻堚枅 Check configuration. Continue (y/n)? " answer
    case $answer in
        [Yy]* ) echo; break;;
        [Nn]* ) echo; exit;;
        * ) echo -e "\nPlease answer (y)es or (n)o.";;
    esac
done

if ! ./ungoogled_packaging/build.sh ; then
    echo '鈻堚枅鈻堚枅 build.sh exited with error'
    exit 1
fi

./ungoogled_packaging/package.sh
ghost picture ghost on 19 Sep 2018

@anchev

Yeah, but as we talked above - it seems not to give the flexibility I am looking for. If it was more customizable then perhaps that would be the right thing to use.

Oops, you're right. I forgot the context.

That is what I would rather be interested to try. Currently looking for info how to do it.

There are two options off the top of my head:

  • Add the bin of the LLVM root directory to your PATH, and ensure that the environment variables specifying the compilers are correctly referencing said LLVM in PATH
  • Remove the GN args that specify to use the "unbundled" compiler, and place the entire LLVM binary tree (i.e. files that would be installed with make install) into third_party/llvm-build/Release+Asserts (which is what is done for macOS and Windows, see their downloads.ini config files)

BTW the whole build process takes about 2 hours here. I wonder if there is a way to optimize it. Any suggestions? I am using a simple bash script for the whole thing:

That's faster than my machine for building Debian stretch, and it isn't even as aggressive with compiler optimizations as Portable Linux...

2-4 hours is pretty typical for higher-end regular consumer machines. If you have a lot of threads, you could try reducing the jumbo file merge limit (default is 50).

Eloston picture Eloston on 19 Sep 2018

@anchev

using what ever i find to get it to build

I am not really a fan of such approach. It makes even less sense along the lines of the current project: one is trying to free oneself from untrusted stuff and the way to it is not to use whatever. (imo)

This is just a step to get it to build to see what it need THEN i clean the install to make a final release this is just to avoid loosing time.

build llvm/clang/lld from sources it's not that big of a deal

That is what I would rather be interested to try. Currently looking for info how to do it.

Here is how to https://clang.llvm.org/get_started.html also have-a-look/apply the patchs used in Akhnazile LLVM8 Repo before building
Also you will need to enable it in PATH as @Eloston already indicated... in my case i just symlink it to /usr/bin

Yes indeed you machine is a fast one, it takes 4h in my laptop... but i don't build there i am using google cloud computing... it takes about 10 min there https://github.com/Eloston/ungoogled-chromium/issues/491

The is also an alternative if you have an other machine locally, you could use its cpu to help building it's Distcc but i did not try it

intika picture intika on 19 Sep 2018

Thank you guys.

ensure that the environment variables specifying the
compilers are correctly referencing said LLVM in
PATH

In which file(s) do I check this please?

Remove the GN args

What are "GN args" and how do I remove them?

you could try reducing the jumbo file merge limit

What is "jumbo file merge limit" and how do I reduce
it?

i am using google cloud computing...

@intika you really have an interesting approach -
going back to Google to build an ungoogled chromium :)

ghost picture ghost on 20 Sep 2018

@anchev

ensure that the environment variables specifying the compilers are correctly referencing said LLVM in PATH
In which file(s) do I check this please?

echo $PATH
https://www.google.com/search?hl=en&q=linux+path
Basically its where any binary is located, this variable is used for any command

What are "GN args" and how do I remove them?

https://www.google.com/search?hl=en&q=%22GN+args%22
https://www.chromium.org/developers/gn-build-configuration
Don't worry too much about them in my test config it build without changing those with a custom llvm.

What is "jumbo file merge limit" and how do I reduce it?

https://www.google.com/search?hl=en&q=%22jumbo_file_merge_limit%22
https://www.google.com/search?hl=en&q=%22jumbo+file+merge+limit%22+chromium
https://chromium.googlesource.com/chromium/src/+show/69.0.3455.3/docs/jumbo.md
https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/ThDAjO7fTro

@intika you really have an interesting approach - going back to Google to build an ungoogled chromium :)

...and on a chromebook lol

Some lecture..
https://www.google.com/search?hl=en&q=%22Trusting+Trust%22

Aah by the way if you are not using http://www.google.com for your searches you can try http://www.qwant.com :D :1st_place_medal: :+1: :1234:

intika picture intika on 20 Sep 2018

@anchev

In which file(s) do I check this please?

There's a bin directory in the LLVM root directory. You need to make sure your PATH environment variable includes it, and then ensure the CC, CXX, LD, etc. variables use the LLVM binaries in bin.

What are "GN args" and how do I remove them?

You should read the Chromium build instructions and related documentation on GN for a more complete picture.

The build script generates an args.gn file that you will need to modify before the invocation of gn.

What is "jumbo file merge limit" and how do I reduce it?

Google has good documentation on Jumbo in the docs/ directory of the Chromium source tree.

you really have an interesting approach - going back to Google to build an ungoogled chromium :)

Even though we probably don't want to use the binaries from Cloud Compute, it's nice to see some real data of the power of cloud-based computing. Who knows, it may be useful if this project continues to grow and reproducible builds become a reality...

However, I am biased. I wouldn't be working as much on this project as I have been if it weren't for the depth of new and fun challenges it offers (and not just engineering-related either). I see peope like @intika playing with these kinds of projects for similar reasons; and they bring forth new energy and perspectives in their pursuit.

Eloston picture Eloston on 20 Sep 2018
👍1

Friends, I made it! Thank you for your help :)

I have built LLVM from source and then used it to build ungoogled chromium (UGC). Here is the little script I wrote for the purpose, in case anyone is interested to use it:

#!/bin/bash

START_TIME=$SECONDS

workdir='/tmp/download'
svnurl='https://llvm.org/svn/llvm-project'
src='llvm-project'
build='llvm-build'
dest='/opt/llvm'

echo "鈻堚枅鈻堚枅 Remove ${workdir}/${src}"
rm -rf "${workdir}/${src}"
mkdir -p "${workdir}/${src}"

echo '鈻堚枅鈻堚枅 Checkout LLVM'
cd "${workdir}/${src}"
svn co -q "${svnurl}/llvm/trunk" llvm

echo '鈻堚枅鈻堚枅 Checkout Clang'
cd "${workdir}/${src}"
cd llvm/tools
svn co -q "${svnurl}/cfe/trunk" clang

echo '鈻堚枅鈻堚枅 Checkout Extra Clang Tools'
cd "${workdir}/${src}"
cd llvm/tools/clang/tools
svn co -q "${svnurl}/clang-tools-extra/trunk" extra

echo '鈻堚枅鈻堚枅 Checkout LLD linker'
cd "${workdir}/${src}"
cd llvm/tools
svn co -q "${svnurl}/lld/trunk" lld

echo '鈻堚枅鈻堚枅 Checkout Polly Loop Optimizer'
cd "${workdir}/${src}"
cd llvm/tools
svn co -q "${svnurl}/polly/trunk" polly

echo '鈻堚枅鈻堚枅 Checkout Compiler-RT (required to build the sanitizers)'
cd "${workdir}/${src}"
cd llvm/projects
svn co -q "${svnurl}/compiler-rt/trunk" compiler-rt

echo '鈻堚枅鈻堚枅 Checkout Libomp (required for OpenMP support)'
cd "${workdir}/${src}"
cd llvm/projects
svn co -q "${svnurl}/openmp/trunk" openmp

echo '鈻堚枅鈻堚枅 Checkout libcxx and libcxxabi'
cd "${workdir}/${src}"
cd llvm/projects
svn co -q "${svnurl}/libcxx/trunk" libcxx
svn co -q "${svnurl}/libcxxabi/trunk" libcxxabi

echo '鈻堚枅鈻堚枅 Get the Test Suite Source Code'
cd "${workdir}/${src}"
cd llvm/projects
svn co -q "${svnurl}/test-suite/trunk" test-suite

echo '鈻堚枅鈻堚枅 Configure and build LLVM and Clang'
cd "${workdir}"
rm -rf "${build}"
mkdir "${build}"
cd "${build}"

export CFLAGS="-O3 -pipe -march=native"
export CXXFLAGS="${CFLAGS}"

cmake -G Ninja \
    -DCMAKE_INSTALL_PREFIX="${dest}" \
    -DCMAKE_BUILD_TYPE=Release \
    "${workdir}/${src}/llvm"

if [[ $? -ne 0 ]]
then
    echo '鈻堚枅鈻堚枅 cmake exited with error'
    exit 1
fi

echo '鈻堚枅鈻堚枅 Running ninja'
if ! ninja -j$(nproc) -C "${workdir}/${build}"; then
    echo '鈻堚枅鈻堚枅 Ninja exited with error'
    exit 1
fi

ELAPSED_TIME=$(($SECONDS - $START_TIME))

hours=$((ELAPSED_TIME / 3600))
seconds=$((ELAPSED_TIME % 3600))
minutes=$((seconds / 60))
seconds=$((seconds % 60))

echo "鈻堚枅鈻堚枅 Elapsed time: $hours hour(s) $minutes minute(s) $seconds second(s)"

Note: Perhaps the -DCMAKE_INSTALL_PREFIX is not needed when ninja is used (and not make). Building LLVM takes about 1 hour here.

I also added the following line to the previous script I shared (the one I use to build UGC itself):

export PATH="/tmp/llvm-build/bin:$PATH"

before the line which runs build.sh.

_First impression:_ UGC feels faster than standard Chromium! I don't know if it is due to it being a custom build for the particular machine but it is great.

Some other things I would like to discuss with you if I may:

  • Along the lines of building a portable version: does /etc/default/chromium matter at all? If not - which is the "master" settings file where one can set default preferences for all users on the system?

  • Spell check doesn't seem to work. Is it because it needs connection to Google?

  • Earlier you mentioned that I need to modify the Inox patches in order to have default settings the way I want them. However looking at them I see they include specific line numbers which can change with any new version of Chrome, so it seems to need a continuous careful work which I am not sure I will have the time to do. Isn't there a simpler way to approach this?

  • chrome://net-internals/#hsts section HSTS/PKP shows a link https://www.ch40m1um.qjz9zk/hsts which made me wonder: Is it possible that due to the UGC's specific replacement of host names the "HSTS preloaded list" of Chrome doesn't work? (which has a certain security implication)

  • While browsing chrome://settings/ I was watching a ss -tuapn on the side and noticed a TCP connection to 104.20.23.46:443. At that time I was on the Language setting. Then I ran rcnetwork restart and tried to reproduce it but I could not. Some minutes later when I opened several chrome://flags tabs it showed up again. Since then I haven't seen it. I have no idea what this host is but considering its location it may be worth checking if it is an attempt of Chrome to connect to something:

[~]: geoiplookup 104.20.23.46
GeoIP Country Edition: US, United States
GeoIP City Edition, Rev 1: US, CA, California, San
  Francisco, 94107, 37.769699, -122.393303, 807, 415
  GeoIP ASNum Edition: AS13335 CloudFlare, Inc.

  • uBlock/uMatrix: Do you happen to know where "Export to cloud storage" actually uploads? When I click it I see no Internet connections but it still seems to work.

  • What benefit has using --set-ipv6-probe-false?

  • Does installing an extension as CRX result in possibility to have it auto-updated from Chrome store? I am just looking for a way to install from file "HTTPS Everywhere" but EFF's site seems to offer only CRX downloads.

Re. the build process:

As you can see in my UGC build script there are 3 sed lines. The first 2 of them insert the specific CFLAGS. The third one modifies the ninja line to use the actual number of cores instead of the default (10). My questions are:

  • Do I need to do the first 2 inserts in the build.sh itself or will it be enough to simply export the CFLAGS in my own bash script?

  • Do you think you could probably include my modification of the build.sh by adding -j$(nproc) to the ninja command? It seems a universal modification which would benefit everyone.

  • I have an old 32-bit laptop for which I am willing to build UGC too. But I am afraid it may be a terribly slow process (if possible at all because it has a small disk and not so much RAM). How can I use the stronger desktop machine to create a binary for the laptop? I wonder if it just a matter of different CFLAGS (and what should they be) or something else? I hope you can shed some light.

ghost picture ghost on 23 Sep 2018
🎉1 👍1

Along the lines of building a portable version: does /etc/default/chromium matter at all? If not - which is the "master" settings file where one can set default preferences for all users on the system?

I don't believe this is modified from regular Chromium for linux_portable bundle, but I may be mistaken.

Is it possible that due to the UGC's specific replacement of host names the "HSTS preloaded list" of Chrome doesn't work?

If that's a link to update the HSTS list, then yes it's broken. Auto-updating the HSTS list from Google would violate this project's primary objective, so we would need to find another way to load it if needed.

The built-in HSTS list works just fine; there was a resolved issue about this a while ago.

Spell check doesn't seem to work. Is it because it needs connection to Google?

Yes, the spell check dictionaries need connections to Google. They can be downloaded manually if I'm not mistaken; there's an open issue about this.

Earlier you mentioned that I need to modify the Inox patches in order to have default settings the way I want them. However looking at them I see they include specific line numbers which can change with any new version of Chrome, so it seems to need a continuous careful work which I am not sure I will have the time to do. Isn't there a simpler way to approach this?

You could create a template profile that contains the settings you want, and just use copies of it for running the browser.

uBlock/uMatrix: Do you happen to know where "Export to cloud storage" actually uploads? When I click it I see no Internet connections but it still seems to work.

There are chrome extension APIs for local storage and cloud/sync storage. They have different size restrictions, but the function identically from a read/write API perspective. Both get stored locally in the profile, but cloud/sync storage gets synced to the browser's signed-in Google account if available. (In ungoogled-chromium's case, this obviously does not happen.)

While browsing chrome://settings/ I was watching a ss -tuapn on the side and noticed a TCP connection to 104.20.23.46:443. At that time I was on the Language setting. Then I ran rcnetwork restart and tried to reproduce it but I could not. Some minutes later when I opened several chrome://flags tabs it showed up again. Since then I haven't seen it. I have no idea what this host is but considering its location it may be worth checking if it is an attempt of Chrome to connect to something:

I don't know. I can't find any domain name associated with that IP, and that IP is not in the source code as 104.20.23.46. It doesn't make sense to me that Google would be using Cloudfare as a CDN either. Are you sure that it is ungoogled-chromium causing it?

What benefit has using --set-ipv6-probe-false?

From my usage, it causes Chromium to not use IPv6 because it thinks the probing for IPv6 returned a negative result.

Does installing an extension as CRX result in possibility to have it auto-updated from Chrome store? I am just looking for a way to install from file "HTTPS Everywhere" but EFF's site seems to offer only CRX downloads.

Doesn't seem like it. There's an open issue about using the update URL in the manifest file for updates.

Do I need to do the first 2 inserts in the build.sh itself or will it be enough to simply export the CFLAGS in my own bash script?

Those compiler and linker flags need to be readable by GN, since they're used in the generation of the ninja files (which have all the flags resolved). gn gen is run explicitly once already, but ninja can also run GN if it's necessary. As long as the variables are visible to both the gn and ninja processes, it should be sufficient.

Do you think you could probably include my modification of the build.sh by adding -j$(nproc) to the ninja command? It seems a universal modification which would benefit everyone.

  1. It's not clear to me where nproc is defined
  2. ninja will automatically set the value for -j depending on the number of thread available (see ninja --help)

I have an old 32-bit laptop for which I am willing to build UGC too. But I am afraid it may be a terribly slow process (if possible at all because it has a small disk and not so much RAM). How can I use the stronger desktop machine to create a binary for the laptop? I wonder if it just a matter of different CFLAGS (and what should they be) or something else? I hope you can shed some light.

I believe you will need to set some GN flags for cross-compilation defined within the GN files of the Chromium source code. I have no experience with them (nor cross-compiling in general), so I can't help you here.

Eloston picture Eloston on 24 Sep 2018
👍1

Thanks again for the answers.

If that's a link to update the HSTS list, then yes it's broken.

It's a link in the description saying "See https://www.ch40m1um.qjz9zk/hsts." but I thought it might be related to what you say, that's why I mentioned it. I don't know how to check if it is broken or not. Testing with a few websites showed me HTTP 307 with an HSTS header but I don't know if this confirms anything.

You could create a template profile that contains the settings you want, and just use copies of it for running the browser.

That's what I do.

Are you sure that it is ungoogled-chromium causing it?

What I am sure is that: while I was testing no other programs were open and I took care to stop other network services which create Internt connections (such as ntpd for example). And it happened while I was playing with the settings. But in ss it did not show up with a process name. So I relate it to UGC just because of all these factors.

From my usage, it causes Chromium to not use IPv6 because it thinks the probing for IPv6 returned a negative result.

I can see what it does but I don't understand the benefit of it. Privacy, security, anything else?

It's not clear to me where nproc is defined

[~]: which nproc
/usr/bin/nproc

see ninja --help

I have seen it while writing my script:

-j N     run N jobs in parallel [default=10, derived from CPUs available]

Still it is not clear how exactly "derived from CPUs [is] available". I can't find any documentation saying much about it. The manual summarizes:

No matter what pools you specify, ninja will never run more concurrent jobs than the default parallelism, or the number of jobs specified on the command line (with -j).

which implies that unless explicitly specified, it will always use the default (10). Hence the suggestion.

ghost picture ghost on 25 Sep 2018

I can see what it does but I don't understand the benefit of it. Privacy, security, anything else?

It was connecting to Google's DNS server before. Some others just redirect it to another DNS server that supports IPv6, but I wanted the ability to control whether I want to use IPv6 or not (there was a situation where I was using a dual stack IPv4/6 network configuration)

which implies that unless explicitly specified, it will always use the default (10). Hence the suggestion.

It's recomputed whenever you run the command. On my machine, it's 6.

Eloston picture Eloston on 25 Sep 2018
👍1

Thanks for explaining.

It was connecting to Google's DNS server before.

Does that mean that it no longer connects to it regardless of using this flag?

It's recomputed whenever you run the command.

I wasn't aware of that because I don't read anywhere about it in the docs. Thanks for the info.

ghost picture ghost on 25 Sep 2018

Does that mean that it no longer connects to it regardless of using this flag?

It no longer makes any sort of request to determine the status of IPv6 probing.

Eloston picture Eloston on 27 Sep 2018

Forgive me, I am not a network expert, so this is a bit difficult for me. All I know is that 'probing' is an action used to determine a state of the network (e.g. ping, nmap) and that IPv6 is a newer protocol, with much larger amount of IP addresses and includes certain improvements. Why one would want to avoid IPv6? I hope you can clarify as for a layman.

ghost picture ghost on 27 Sep 2018

Forgive me, I am not a network expert, so this is a bit difficult for me. All I know is that 'probing' is an action used to determine a state of the network (e.g. ping, nmap) and that IPv6 is a newer protocol, with much larger amount of IP addresses and includes certain improvements. Why one would want to avoid IPv6? I hope you can clarify as for a layman.

IPv6 is a whole new world compared to ipv4... i personally don't like it i have my reasons (how this new protocol works, who invented it, and the history...)... now when it come to general IT here is some reason where you may want to disable ipv6

  1. Firewall : if your firewall does not handle ipv6

  2. Setup : if you have a setup that does not handle ipv6 let say an ipv4 VPN for example

  3. A lot of network tool does not handle ipv6 yet..

  4. Security and Stability : even if ipv6 have been around since a quite long time it still have some major difference with ipv4, some links http://www.ipv6now.com.au/primers/IPv6SecurityIssues.php and https://www.google.com/search?hl=en&q=leak+ipv6 and https://www.google.com/search?hl=en&q=vulnerability+ipv6

  5. Complexity : many IT prefer ipv4 for its simple addressing system, compared to v6 when you look at an ip it a little bit more complex to handle and understand

... and so on we could find a million reasons not to use it... the main purpose of ipv6 is the limited amount of available ips addresses and for the moment the impact is limited...

intika picture intika on 27 Sep 2018

Thanks @intika. I wasn't aware of 4. Now I have another thing to worry about :)

ghost picture ghost on 28 Sep 2018

@anchev

Forgive me, I am not a network expert, so this is a bit difficult for me. All I know is that 'probing' is an action used to determine a state of the network

Sorry, I wasn't clear enough. There is a small routine in Chromium that probes the availability of IPv6 via pinging a IPv6 DNS. I stubbed that method to force the result of the probing routine to be determined only by that command-line flag.

Why one would want to avoid IPv6? I hope you can clarify as for a layman.

I'm not nearly as well versed in IPv6 implications as @intika. I only have surface-level knowledge of it.

Eloston picture Eloston on 28 Sep 2018

Thanks @Eloston.

So in short (as far as I understand): it is better to use this CLI option unless there is some special reason not to use it (e.g. accessing a web site which uses only an IPv6 addres? any other?)

ghost picture ghost on 28 Sep 2018

So in short (as far as I understand): it is better to use this CLI option unless there is some special reason not to use it (e.g. accessing a web site which uses only an IPv6 addres? any other?)

There's no harm in enabling it if you don't need IPv6. But if you disable IPv6 in your system, the CLI flag won't affect much.

Eloston picture Eloston on 29 Sep 2018

To answer my own question:

Along the lines of building a portable version: does /etc/default/chromium matter at all?

I tested today and it seems this file is disregarded.

ghost picture ghost on 30 Sep 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

lipici picture lipici  路  3Comments
ribatamu picture ribatamu  路  3Comments
Chilcout picture Chilcout  路  3Comments
tonowoe picture tonowoe  路  3Comments
playgithub picture playgithub  路  3Comments