Straight.el: straight-vc-git-default-clone-depth pulls all branches, contrary to the documentation

[jolyon-wright]

straight-vc-git-default-clone-depth pulls all branches, contrary to the documentation

I have forked the repo and resolved the issue so it behaves as I want. My fork is here: https://github.com/jolyon-wright/straight.el . It's a one liner!

Straight is great btw; I have been using it for a number of years.

Directions to reproduce

Here is the behaviour on the master and develop branches of straight.

• use the init.el below
• start emacs
• observe which branches have been pulled

  ;;; init.el --- repro straight-vc-git-default-clone-depth oddity -*- lexical-binding: t -*- 

  (setq straight-repository-branch "develop") 
  (defvar bootstrap-version) 

  ;; straight-vc-git-default-clone-depth help:- 
  ;; 
  ;; " 
  ;; The value should be the symbol ‘full’ or an integer. If the value 
  ;; is ‘full’, clone the whole history of repositories. If the value 
  ;; is an integer N, remote repositories are cloned with the options 
  ;; --depth N --single-branch --no-tags. 
  ;; " 
  ;;.. hm... actually it clones with --no-single-branch 

  (setq straight-vc-git-default-clone-depth 1) 

  (let ((bootstrap-file 
         (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory)) 
        (bootstrap-version 5)) 
    (unless (file-exists-p bootstrap-file) 

      ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 
      ;; please see notes below
      ;;
      ;; (setq straight-recipe-overrides 
      ;;        '((nil . ((straight :host github 
      ;;                            :repo "jolyon-wright/straight.el" 
      ;;                            :branch "master" 
      ;;                            :files ("straight.el")))))) 
      ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 

      (with-current-buffer 
          (url-retrieve-synchronously 
           "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el" 
           'silent 'inhibit-cookies) 
        (goto-char (point-max)) 
        (eval-print-last-sexp))) 
    (load bootstrap-file nil 'nomessage)) 

  (setq straight-use-package-by-default t) 
  (straight-use-package 'use-package) 

  ;; when emacs is up all the branches have been pulled:- 


  ;; jolyon@Jolyons-MacBook-Pro use-package % git branch -a 
  ;; git branch -a 
  ;; * master 
  ;;   remotes/origin/HEAD -> origin/master 
  ;;   remotes/origin/gh-pages 
  ;;   remotes/origin/master 
  ;;   remotes/origin/next 
  ;;   remotes/origin/revert-617-lexical-bindings 
  ;;   remotes/origin/revert-645-fix-report 
  ;;   remotes/origin/split-bind-key 
  ;; jolyon@Jolyons-MacBook-Pro use-package % pwd 
  ;; /Users/jolyon/.emacs.d/straight/repos/use-package 
  ;; jolyon@Jolyons-MacBook-Pro use-package %  


  ;; plugging in the fork by uncommenting the recipe overide and using 
  ;; the https://github.com/jolyon-wright/straight.el fork gives me the behavour. 
  ;; I want...

  ;; when the override is uncommented and emacs is relaunched I see this:- 

  ;; jolyon@Jolyons-MacBook-Pro use-package % git branch -a 
  ;; git branch -a 
  ;; * master 
  ;;   remotes/origin/HEAD -> origin/master 
  ;;   remotes/origin/master 
  ;; jolyon@Jolyons-MacBook-Pro use-package %  

Version information

• Emacs version: 27.2
• Operating system: Macos 11.2.3 (also reproduced on ubuntu 2004 and Windows 10)

[progfolio]

Thank you for the thorough report.
You're correct, the docstring for straight-vc-git-default-clone-depth is misleading.
The rationale for the --no-single-branch behavior is discussed here:

578

I will update the docstring to reflect that.

[progfolio]

Out of curiosity, what's your reason for only wanting a single branch? We may be able to add an additional option or code path to support the use case.

[jolyon-wright]

Thanks progfolio; if this could be added to straight my life would be much easier... I am not comfortable using my fork to achieve this!

But I do understand that adding this somewhat strange option may make the lives of others a bit more baffling...

The reason I only want to clone a single branch is because my emacs configuration contains emacs (!!!!)... yes; you read that right :)

If I am pulling my configuration on windows there is a branch that contains the emacs binaries, hunspell and all that Windows strangeness so I don't need to install anything on a windows box except git. This lives in its own branch.

This enables me to have a consistent emacs experience regardless of the emacs backend OS.

However on Linux or Mac I absolutely do not want to pull that particular branch because the lights will dim! ie - it's huge...

I do understand this is a bit of an offbeat way of doing things -- the work you folk have put into straight means I can kind of get there one way or another; so a big thanks!!

jolyon

[raxod502]

I don't see any reason we can't make the interface for straight-vc-git-default-clone-depth a bit more advanced, so that in addition to the currently supported values (N or full), we would also support a list like (N) or (full) or (N single-branch) or (full single-branch), with other properties able to be added in future.

[jolyon-wright]

That would be great; I appreciate that the way I am abusing straight is a bit of an edge case but a more general justification for doing this might be simply decreasing download size.

This would make life a lot easier for users with poor internet connection!

[progfolio]

we would also support a list like (N) or (full) or (N single-branch) or (full single-branch), with other properties able to be added in future

Done in #744

@jolyon-wright:
Would you mind testing this out?
You should be able to use the :depth keyword in your Emacs recipe as described above on the latest develop branch.

[jolyon-wright]

thanks... sigh... I might need some guidance on getting it to work.... would you expect this to give me a single branch clone?

;; -*- lexical-binding: t; -*-

(setq straight-repository-branch "develop")

(defvar bootstrap-version)
(setq straight-vc-git-default-clone-depth '(1 single-branch))

(let ((bootstrap-file
;; et c

I'm getting a really forbidding backtrace... I think I am misunderstanding the comment about passing a list!

[progfolio]

It's possible I've missed a corner case. May I see the backtrace?

[jolyon-wright]

Thanks,

Here is the backtrace:-

Debugger entered--Lisp error: (error "straight.el bootstrap failed: Invalid value (1 sin...")
  signal(error ("straight.el bootstrap failed: Invalid value (1 sin..."))
  error("straight.el bootstrap failed: %s" "Invalid value (1 single-branch) of depth for https...")
  (if (= 0 (call-process (let ((emacs-binary-path (expand-file-name invocation-name invocation-directory)) (runemacs-binary-path (if (straight--windows-os-p) (progn ...)))) (if (and runemacs-binary-path (file-exists-p runemacs-binary-path)) runemacs-binary-path emacs-binary-path)) nil '(t t) nil "--batch" "--no-window-system" "--quick" "--load" temp-file)) nil (error "straight.el bootstrap failed: %s" (buffer-string)))
  (progn (if (= 0 (call-process (let ((emacs-binary-path (expand-file-name invocation-name invocation-directory)) (runemacs-binary-path (if ... ...))) (if (and runemacs-binary-path (file-exists-p runemacs-binary-path)) runemacs-binary-path emacs-binary-path)) nil '(t t) nil "--batch" "--no-window-system" "--quick" "--load" temp-file)) nil (error "straight.el bootstrap failed: %s" (buffer-string))))
  (unwind-protect (progn (if (= 0 (call-process (let ((emacs-binary-path ...) (runemacs-binary-path ...)) (if (and runemacs-binary-path ...) runemacs-binary-path emacs-binary-path)) nil '(t t) nil "--batch" "--no-window-system" "--quick" "--load" temp-file)) nil (error "straight.el bootstrap failed: %s" (buffer-string)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if (= 0 (call-process (let (... ...) (if ... runemacs-binary-path emacs-binary-path)) nil '(t t) nil "--batch" "--no-window-system" "--quick" "--load" temp-file)) nil (error "straight.el bootstrap failed: %s" (buffer-string)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
  (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if (= 0 (call-process (let ... ...) nil '... nil "--batch" "--no-window-system" "--quick" "--load" temp-file)) nil (error "straight.el bootstrap failed: %s" (buffer-string)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
  (let ((temp-file (make-temp-file "straight.el~"))) (write-region nil nil temp-file nil 'silent) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if (= 0 (call-process ... nil ... nil "--batch" "--no-window-system" "--quick" "--load" temp-file)) nil (error "straight.el bootstrap failed: %s" (buffer-string)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))
  (save-current-buffer (set-buffer (url-retrieve-synchronously (format (concat "https://raw.githubusercontent.com/" "raxod502/straight.el/install/%s/straight.el") (substring (symbol-name version) 1)) 'silent 'inhibit-cookies)) (if (equal url-http-response-status 200) nil (error "Unknown recipe version: %S" version)) (delete-region (point-min) url-http-end-of-headers) (goto-char (point-min)) (princ ";; -*- coding: utf-8 -*-" (current-buffer)) (print (cons 'progn (let* ((vars nil) (regexps '("bootstrap-version" "straight-[a-z-]+" "user-emacs-directory")) (regexp (format "^\\(%s\\)$" (mapconcat ... regexps "\\|")))) (mapatoms #'(lambda (sym) (if ... ...))) (mapcar #'(lambda (var) (list ... var ...)) vars))) (current-buffer)) (goto-char (point-max)) (print '(progn (straight-use-package-no-build `(straight :type git :host github :repo ,... :branch ,...)) (unless (and (boundp 'bootstrap-version) (integerp bootstrap-version) (>= bootstrap-version 3)) (let* ((recipe ...) (local-repo ...) (link-target ...) (link-name ...)) (ignore-errors (delete-file link-name)) (if (bound-and-true-p straight-use-symlinks) (if ... ... ...) (with-temp-file link-name ...))))) (current-buffer)) (let ((temp-file (make-temp-file "straight.el~"))) (write-region nil nil temp-file nil 'silent) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if (= 0 ...) nil (error "straight.el bootstrap failed: %s" ...))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))))
  (let ((version nil) (straight-profiles (if (boundp 'straight-profiles) straight-profiles '((nil . "default.el")))) (straight-install-dir (or (and (boundp 'straight-base-dir) straight-base-dir) user-emacs-directory)) (print-length nil) (print-level nil)) (let ((--dolist-tail-- (mapcar #'cdr straight-profiles))) (while --dolist-tail-- (let ((lockfile-name (car --dolist-tail--))) (let ((lockfile-path (concat straight-install-dir "straight/versions/" lockfile-name))) (if (file-exists-p lockfile-path) (progn (condition-case nil ... ...)))) (setq --dolist-tail-- (cdr --dolist-tail--))))) (if version nil (setq version :beta)) (save-current-buffer (set-buffer (url-retrieve-synchronously (format (concat "https://raw.githubusercontent.com/" "raxod502/straight.el/install/%s/straight.el") (substring (symbol-name version) 1)) 'silent 'inhibit-cookies)) (if (equal url-http-response-status 200) nil (error "Unknown recipe version: %S" version)) (delete-region (point-min) url-http-end-of-headers) (goto-char (point-min)) (princ ";; -*- coding: utf-8 -*-" (current-buffer)) (print (cons 'progn (let* ((vars nil) (regexps '...) (regexp (format "^\\(%s\\)$" ...))) (mapatoms #'(lambda ... ...)) (mapcar #'(lambda ... ...) vars))) (current-buffer)) (goto-char (point-max)) (print '(progn (straight-use-package-no-build `(straight :type git :host github :repo ... :branch ...)) (unless (and (boundp ...) (integerp bootstrap-version) (>= bootstrap-version 3)) (let* (... ... ... ...) (ignore-errors ...) (if ... ... ...)))) (current-buffer)) (let ((temp-file (make-temp-file "straight.el~"))) (write-region nil nil temp-file nil 'silent) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (if ... nil ...)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))))
  (progn (message "Bootstrapping straight.el...") (let ((min-version "25.1")) (if (version< emacs-version min-version) (progn (error (concat "straight.el requires at least Emacs %s, " "but you are running Emacs %s") min-version emacs-version)))) (if (boundp 'straight-repository-branch) (progn (if (stringp straight-repository-branch) nil (error "The `straight-repository-branch' must be a string ..." straight-repository-branch)))) (if (executable-find "git") nil (user-error "Git executable not found. straight.el requires git")) (require 'cl-lib) (require 'url-http) (defvar url-http-end-of-headers) (defvar url-http-response-status) (defalias 'straight--windows-os-p #'(lambda nil "Check if the current operating system is Windows." (memq system-type '(ms-dos windows-nt)))) (let ((version nil) (straight-profiles (if (boundp 'straight-profiles) straight-profiles '((nil . "default.el")))) (straight-install-dir (or (and (boundp 'straight-base-dir) straight-base-dir) user-emacs-directory)) (print-length nil) (print-level nil)) (let ((--dolist-tail-- (mapcar #'cdr straight-profiles))) (while --dolist-tail-- (let ((lockfile-name (car --dolist-tail--))) (let ((lockfile-path ...)) (if (file-exists-p lockfile-path) (progn ...))) (setq --dolist-tail-- (cdr --dolist-tail--))))) (if version nil (setq version :beta)) (save-current-buffer (set-buffer (url-retrieve-synchronously (format (concat "https://raw.githubusercontent.com/" "raxod502/straight.el/install/%s/straight.el") (substring (symbol-name version) 1)) 'silent 'inhibit-cookies)) (if (equal url-http-response-status 200) nil (error "Unknown recipe version: %S" version)) (delete-region (point-min) url-http-end-of-headers) (goto-char (point-min)) (princ ";; -*- coding: utf-8 -*-" (current-buffer)) (print (cons 'progn (let* ((vars nil) (regexps ...) (regexp ...)) (mapatoms #'...) (mapcar #'... vars))) (current-buffer)) (goto-char (point-max)) (print '(progn (straight-use-package-no-build `...) (unless (and ... ... ...) (let* ... ... ...))) (current-buffer)) (let ((temp-file (make-temp-file "straight.el~"))) (write-region nil nil temp-file nil 'silent) (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn ...) (and ... ...))))))) (message "Bootstrapping straight.el...done"))
  (progn (progn (message "Bootstrapping straight.el...") (let ((min-version "25.1")) (if (version< emacs-version min-version) (progn (error (concat "straight.el requires at least Emacs %s, " "but you are running Emacs %s") min-version emacs-version)))) (if (boundp 'straight-repository-branch) (progn (if (stringp straight-repository-branch) nil (error "The `straight-repository-branch' must be a string ..." straight-repository-branch)))) (if (executable-find "git") nil (user-error "Git executable not found. straight.el requires git")) (require 'cl-lib) (require 'url-http) (defvar url-http-end-of-headers) (defvar url-http-response-status) (defalias 'straight--windows-os-p #'(lambda nil "Check if the current operating system is Windows." (memq system-type '(ms-dos windows-nt)))) (let ((version nil) (straight-profiles (if (boundp 'straight-profiles) straight-profiles '(...))) (straight-install-dir (or (and (boundp ...) straight-base-dir) user-emacs-directory)) (print-length nil) (print-level nil)) (let ((--dolist-tail-- (mapcar #'cdr straight-profiles))) (while --dolist-tail-- (let ((lockfile-name ...)) (let (...) (if ... ...)) (setq --dolist-tail-- (cdr --dolist-tail--))))) (if version nil (setq version :beta)) (save-current-buffer (set-buffer (url-retrieve-synchronously (format (concat "https://raw.githubusercontent.com/" "raxod502/straight.el/install/%s/straight.el") (substring ... 1)) 'silent 'inhibit-cookies)) (if (equal url-http-response-status 200) nil (error "Unknown recipe version: %S" version)) (delete-region (point-min) url-http-end-of-headers) (goto-char (point-min)) (princ ";; -*- coding: utf-8 -*-" (current-buffer)) (print (cons 'progn (let* (... ... ...) (mapatoms ...) (mapcar ... vars))) (current-buffer)) (goto-char (point-max)) (print '(progn (straight-use-package-no-build ...) (unless ... ...)) (current-buffer)) (let ((temp-file (make-temp-file "straight.el~"))) (write-region nil nil temp-file nil 'silent) (let ((temp-buffer ...)) (save-current-buffer (set-buffer temp-buffer) (unwind-protect ... ...)))))) (message "Bootstrapping straight.el...done")))
  eval((progn (progn (message "Bootstrapping straight.el...") (let ((min-version "25.1")) (if (version< emacs-version min-version) (progn (error (concat "straight.el requires at least Emacs %s, " "but you are running Emacs %s") min-version emacs-version)))) (if (boundp 'straight-repository-branch) (progn (if (stringp straight-repository-branch) nil (error "The `straight-repository-branch' must be a string ..." straight-repository-branch)))) (if (executable-find "git") nil (user-error "Git executable not found. straight.el requires git")) (require 'cl-lib) (require 'url-http) (defvar url-http-end-of-headers) (defvar url-http-response-status) (defalias 'straight--windows-os-p #'(lambda nil "Check if the current operating system is Windows." (memq system-type '...))) (let ((version nil) (straight-profiles (if (boundp ...) straight-profiles '...)) (straight-install-dir (or (and ... straight-base-dir) user-emacs-directory)) (print-length nil) (print-level nil)) (let ((--dolist-tail-- (mapcar ... straight-profiles))) (while --dolist-tail-- (let (...) (let ... ...) (setq --dolist-tail-- ...)))) (if version nil (setq version :beta)) (save-current-buffer (set-buffer (url-retrieve-synchronously (format ... ...) 'silent 'inhibit-cookies)) (if (equal url-http-response-status 200) nil (error "Unknown recipe version: %S" version)) (delete-region (point-min) url-http-end-of-headers) (goto-char (point-min)) (princ ";; -*- coding: utf-8 -*-" (current-buffer)) (print (cons 'progn (let* ... ... ...)) (current-buffer)) (goto-char (point-max)) (print '(progn ... ...) (current-buffer)) (let ((temp-file ...)) (write-region nil nil temp-file nil 'silent) (let (...) (save-current-buffer ... ...))))) (message "Bootstrapping straight.el...done"))) t)
  elisp--eval-last-sexp(t)
  eval-last-sexp(t)
  eval-print-last-sexp()
  (save-current-buffer (set-buffer (url-retrieve-synchronously "https://raw.githubusercontent.com/raxod502/straigh..." 'silent 'inhibit-cookies)) (goto-char (point-max)) (eval-print-last-sexp))
  (if (file-exists-p bootstrap-file) nil (save-current-buffer (set-buffer (url-retrieve-synchronously "https://raw.githubusercontent.com/raxod502/straigh..." 'silent 'inhibit-cookies)) (goto-char (point-max)) (eval-print-last-sexp)))
  (let ((bootstrap-file (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory)) (bootstrap-version 5)) (if (file-exists-p bootstrap-file) nil (save-current-buffer (set-buffer (url-retrieve-synchronously "https://raw.githubusercontent.com/raxod502/straigh..." 'silent 'inhibit-cookies)) (goto-char (point-max)) (eval-print-last-sexp))) (load bootstrap-file nil 'nomessage))
  eval-buffer(#<buffer  *load*> nil "/home/jw/.emacs.d/init.el" nil t)  ; Reading at buffer position 1025
  load-with-code-conversion("/home/jw/.emacs.d/init.el" "/home/jw/.emacs.d/init.el" t t)
  load("/home/jw/.emacs.d/init" noerror nomessage)
  startup--load-user-init-file(#f(compiled-function () #<bytecode 0x155d6bbcf851>) #f(compiled-function () #<bytecode 0x155d6bbcf819>) t)
  command-line()
  normal-top-level()

Here is my init.el:-

;; -*- lexical-binding: t; -*-

(setq straight-repository-branch "develop")

(defvar bootstrap-version)
;;(setq straight-vc-git-default-clone-depth 1) ;; the strange fork single branches this
(setq straight-vc-git-default-clone-depth '(1 single-branch))

(let ((bootstrap-file
       (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory))
      (bootstrap-version 5))
  (unless (file-exists-p bootstrap-file)
    ;;
    ;; farewell strange fork
    ;;
    ;; (setq straight-recipe-overrides
    ;;       '((nil . ((straight :host github
    ;;                           :repo "jolyon-wright/straight.el"
    ;;                           :branch "master"
    ;;                           :files ("straight.el"))))))
    (with-current-buffer
        (url-retrieve-synchronously
         "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
         'silent 'inhibit-cookies)
      (goto-char (point-max))
      (eval-print-last-sexp)))
  (load bootstrap-file nil 'nomessage))

(setq straight-use-package-by-default t)
(straight-use-package 'use-package)

I also have an early-init.el:-

;;; early-init.el --- early bird  -*- no-byte-compile: t -*-


(setq package-enable-at-startup nil)

;;; early-init.el ends here

I'm trying to think of anything else that could be helpful... this is on ubuntu 1804 with emacs 27.2...

cheers,

jolyon

[progfolio]

Thanks for the info. Try setting straight-vc-git-default-clone-depth just after the bootstrap snippet.

;; -*- lexical-binding: t; -*-

(setq straight-repository-branch "develop")
(defvar bootstrap-version)

(let ((bootstrap-file
       (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory))
      (bootstrap-version 5))
  (unless (file-exists-p bootstrap-file)
    (with-current-buffer
        (url-retrieve-synchronously
         "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
         'silent 'inhibit-cookies)
      (goto-char (point-max))
      (eval-print-last-sexp)))
  (load bootstrap-file nil 'nomessage))

(setq straight-vc-git-default-clone-depth '(1 single-branch))
(setq straight-use-package-by-default t)
(straight-use-package 'use-package)

I believe that will work. I'll look into this more to get it working so that you can set it in either location, too.

Related: #302

[jolyon-wright]

it does!

Obviously I get multiple branches of straight; so this is working for me now:-

;; -*- lexical-binding: t; -*-

(setq straight-repository-branch "develop")
(setq straight-vc-git-default-clone-depth 1) ;; this gives me a single branch of https://github.com/raxod502/straight.el.git
(defvar bootstrap-version)

(let ((bootstrap-file
       (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory))
      (bootstrap-version 5))
  (unless (file-exists-p bootstrap-file)
    (with-current-buffer
        (url-retrieve-synchronously
         "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
         'silent 'inhibit-cookies)
      (goto-char (point-max))
      (eval-print-last-sexp)))
  (load bootstrap-file nil 'nomessage))

(setq straight-vc-git-default-clone-depth '(1 single-branch)) ;; the new idiom!
(setq straight-use-package-by-default t)
(straight-use-package 'use-package)

[raxod502]

It should be possible to let this user option affect the bootstrap itself, so that you'd get a shallow clone of straight.el (and not just all other repos). To do that, we'd need to update the lockfile version keyword:

https://github.com/raxod502/straight.el/blob/253d7db6c1165741208219e7b3a29beb12dcd731/straight.el#L6214-L6226

This will mean that if you've specified a more recent version of straight.el in your lockfile, or there is no lockfile, then a newer version of straight.el will be fetched and used to bootstrap the installation.