Taichi: Avoid `import taichi as ti` for operations like `ti format`

[yuanming-hu]

Concisely describe the proposed feature
As mentioned by @k-ye we might consider removing the dependency on taichi_core.so for ti format. This can lead to faster execution time, and more robustness (if taichi_core.so has load-time linking errors we should still be able to format our code)

Describe the solution you'd like (if any)
Just avoid importing taichi.

[k-ye]

I can take a look of this

[archibate]

Btw, will ti format behave well if $TAICHI_REPO_DIR not set? Will it consider ~/.taichi as a git repo?

[archibate]

Maybe we should move ti.core.format into ti.misc.format? Since it's not core at all, non-devs won't want that. Also importing ti.core will load the libtaichi_core.so.

[archibate]

Also ti update, ti build won't want import taichi_core. We should not make
https://github.com/taichi-dev/taichi/blob/6f8c1fe72a9607a3d893f610cc271123b33cffa5/python/taichi/core/util.py#L265
hardcoded in util.py anymore.
But in somewhere like ti.core.do_startup_works().

[archibate]

https://github.com/taichi-dev/taichi/blob/6f8c1fe72a9607a3d893f610cc271123b33cffa5/python/taichi/core/util.py#L241
Can we have ninja -C build for choice? I guess ti build is only useful for win...

[k-ye]

ti.format

I don't know, haven't take a closer look yet. Feel free to take this over if you're more interested in it..

ninja

While I like ninja + GN for its simplicity (TBH I never learnt much about CMake's syntax), i guess it's quite a huge decision to switch to another build system. But I guess you're simply generating ninja files from CMake? Out of curiosity, how much build performance gain do you see when using ninja vs. make?

I saw that you have lots of customized pieces in your own workflow, maybe it's time to consider some rudimentary support for .taichi?

[archibate]

According to http://hamelot.io/programming/make-vs-ninja-performance-comparison/, Ninja is good when changeset is small:

Ninja and Make are very close in terms of performance. It has less overhead when performing incremental builds but for full builds any performance gains would be negligible.

I saw that you have lots of customized pieces in your own workflow, maybe it's time to consider some rudimentary support for .taichi?

Sounds reasonable, but different devs have different hopes, note sure if .taichi can cover them all..
e.g. TAICHI_REPO_DIR and some personal aliases to .bash_localrc, and added .*_localrc to .gitignore_localrc.

(ci) [bate@archit taichi]$ cat .bash_localrc 
export TAICHI_REPO_DIR=/home/bate/Develop/taichi
export PYTHONPATH=$TAICHI_REPO_DIR/python:$PYTHONPATH
export PATH=$TAICHI_REPO_DIR/bin:/opt/llvm70/bin:$PATH

git-branch-prompt() {
  local branch=`git symbolic-ref HEAD 2> /dev/null | cut -d/ -f3-`
  [ ! $branch ] || echo "($branch) "
}

alias nc='ninja -C build'
alias np='ninja -C build && python'
alias recmake='cd `readlink build`; cmake $TAICHI_REPO_DIR -G Ninja; cd $TAICHI_REPO_DIR'
PS1='$(git-branch-prompt)'$PS1
alias g='git'
alias gs='git s'
alias gl='git l'
alias gc='git c'
alias gm='git m'
alias ga='git a'
alias gd='git d'
alias gdc='git dc'
alias gp='git p'
alias gpf='git pf'
alias gps='git ps'
alias gq='git q'
alias gg='git g'
alias ggo='git go'
(ci) [bate@archit taichi]$ cat ~/.bash_localrc 
t() {
  mkdir -p /tmp/taichi/build
  ln -sf /tmp/taichi/build ~/Develop/taichi/
  cd ~/Develop/taichi
  . .bash_localrc
  cd /tmp/taichi/build
  test -d CMakeFiles || cmake ~/Develop/taichi -G Ninja
  cd ~/Develop/taichi
  bashes
}

maybe they already have git-branch-prompt... and g a... not all people has that git config...

[archibate]

I personally think ti.format should be put in a external script rather than in core/util.py, we can't stop python from import taichi when import taichi.core.util...

[yuanming-hu]

I personally think ti.format should be put in a external script rather than in core/util.py, we can't stop python from import taichi when import taichi.core.util...

Right, as you said, it's not really core...

[archibate]

Or we can load-on-request, only load taichi_core.so when first ti.init is called. With this attempt, no core.util modification required.

[k-ye]

Sounds reasonable, but different devs have different hopes, note sure if .taichi can cover them all..

.taichi is not meant to cover them all. Rather, it should only cover taichi relevant parts..

For example, you pointed out whether we can use ninja here:

https://github.com/taichi-dev/taichi/blob/6f8c1fe72a9607a3d893f610cc271123b33cffa5/python/taichi/core/util.py#L241

That's part of Taichi, so it should be configurable inside .taichi.

Another example is the repo settings, where you would need to customize upstream and origin in order for ti format to work properly. I think .taichi should only care about things that are configurable inside ti. Other things in *_localrc are just non-Taichi relevant things.

[archibate]

For example, you pointed out whether we can use ninja here:

Like make_command: ninja -C build in .taichi/util.yml? I would rather simply type ninja -C build instead of ti build.

Also found this may apply to TI_ARCH, TI_LOGGING_LEVEL, see #522 #519.

[k-ye]

i failed to get your point... If you prefer just using ninja -C build, why would you suggest changing the impl of ti build ...

[archibate]

i failed to get your point... If you prefer just using ninja -C build, why would you suggest changing the impl of ti build ...

I mean, I can type nc for alias of that, but CI may want that and let cmake generate for ninja.. anyway, forget what I just said..

[k-ye]

OK... Again, you seem to have more opinions and ideas on. this. So if you feel more interested, feel free to take it over. Otherwise I'll probably just do some clean up here, like what you said in https://github.com/taichi-dev/taichi/issues/647#issuecomment-603325844

[github-actions[bot]]

Warning: The issue has been out-of-update for 50 days, marking stale.

[archibate]

I'm on this. Using the misc/format.py approach, thx!