Syncers

Overview

Paludis makes use of scripts called 'syncers' to perform certain sync actions (e.g. handling paludis --sync for e format repositories). Some syncers support additional options that can be set via the sync_options repository configuration key. Syncers can be added for additional protocols (for example, if you need to use a repository that uses a weird version control system), and built-in syncers can be replaced (for example, if you want to use a non-standard program).

The syncer protocol used by Paludis is not always identical to the one used by the syncing program. In particular, several syncing programs support URIs in the form http://, and Paludis would not be able to tell whether such a URI should be synced by, say, Subversion, Git or Darcs. Instead, you must use svn+http:// and so on.

Standard Syncers

Paludis ships with the following sync protocols (you can use cave print-sync-protocols and cave sync-protocol-options to get the list yourself). Note that many of these syncers depend upon external programs that are not listed as dependencies for Paludis.


bzr
    URL syntax: bzr://SERVER/PATH
    Options:
        --bzr-checkout-option=OPTION       Pass OPTION to bzr checkout
        --bzr-switch-option=OPTION         Pass OPTION to bzr switch

bzr+aftp
    URL syntax: bzr+aftp://[USERNAME[:PASSWORD]@]SERVER/PATH
    Options:
        --bzr-checkout-option=OPTION       Pass OPTION to bzr checkout
        --bzr-switch-option=OPTION         Pass OPTION to bzr switch

bzr+file
    URL syntax: bzr+file:///PATH
    Options:
        --bzr-checkout-option=OPTION       Pass OPTION to bzr checkout
        --bzr-switch-option=OPTION         Pass OPTION to bzr switch

bzr+ftp
    URL syntax: bzr+ftp://[USERNAME[:PASSWORD]@]SERVER/PATH
    Options:
        --bzr-checkout-option=OPTION       Pass OPTION to bzr checkout
        --bzr-switch-option=OPTION         Pass OPTION to bzr switch

bzr+http
    URL syntax: bzr+http://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --bzr-checkout-option=OPTION       Pass OPTION to bzr checkout
        --bzr-switch-option=OPTION         Pass OPTION to bzr switch

bzr+https
    URL syntax: bzr+https://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --bzr-checkout-option=OPTION       Pass OPTION to bzr checkout
        --bzr-switch-option=OPTION         Pass OPTION to bzr switch

bzr+lp
    URL syntax: bzr+lp://PROJECT
    Options:
        --bzr-checkout-option=OPTION       Pass OPTION to bzr checkout
        --bzr-switch-option=OPTION         Pass OPTION to bzr switch

bzr+sftp
    URL syntax: bzr+sftp://[USERNAME[:PASSWORD]@]SERVER/PATH
    Options:
        --bzr-checkout-option=OPTION       Pass OPTION to bzr checkout
        --bzr-switch-option=OPTION         Pass OPTION to bzr switch

bzr+ssh
    URL syntax: bzr+ssh://[USERNAME[:PASSWORD]@]SERVER/PATH
    Options:
        --bzr-checkout-option=OPTION       Pass OPTION to bzr checkout
        --bzr-switch-option=OPTION         Pass OPTION to bzr switch

cvs+ext
    URL syntax: cvs+ext://[USERNAME[:PASSWORD]@]SERVER:/PATH:MODULE
    Options:
        --cvs-option=OPTION             Pass OPTION to cvs as a global option
        --cvs-checkout-option=OPTION    Pass OPTION to cvs checkout

cvs+pserver
    URL syntax: cvs+pserver://[USERNAME[:PASSWORD]@]SERVER:/PATH:MODULE
    Options:
        --cvs-option=OPTION             Pass OPTION to cvs as a global option
        --cvs-checkout-option=OPTION    Pass OPTION to cvs checkout

cvs+ssh
    URL syntax: cvs+ssh://[USERNAME[:PASSWORD]@]SERVER:/PATH:MODULE
    Options:
        --cvs-option=OPTION             Pass OPTION to cvs as a global option
        --cvs-checkout-option=OPTION    Pass OPTION to cvs checkout

darcs+file
    URL syntax: darcs+file:///PATH
    Options:
        --darcs-get-option=OPTION       Pass OPTION to darcs get
        --darcs-pull-option=OPTION      Pass OPTION to darcs pull

darcs+http
    URL syntax: darcs+http://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --darcs-get-option=OPTION       Pass OPTION to darcs get
        --darcs-pull-option=OPTION      Pass OPTION to darcs pull

darcs+https
    URL syntax: darcs+https://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --darcs-get-option=OPTION       Pass OPTION to darcs get
        --darcs-pull-option=OPTION      Pass OPTION to darcs pull

darcs+ssh
    URL syntax: darcs+ssh://[USERNAME[:PASSWORD]@]SERVER:/PATH
    Options:
        --darcs-get-option=OPTION       Pass OPTION to darcs get
        --darcs-pull-option=OPTION      Pass OPTION to darcs pull

dummy
    URL syntax: dummy://whatever/
    Options:
        --count-to=n                    Delay noisily for n seconds
    Notes:
        This syncer creates a single timestamp file in the target directory. It
        has no practical purpose, but it can be useful for testing.

file
    URL syntax: file:///PATH
    Options:
        --exclude=PATTERN               Use PATTERN as an exclude pattern
        --exclude-from=FILE             Use FILE as a list of exclude patterns
        --rsync-option=OPTION           Pass OPTION to rsync

git
    URL syntax: git://SERVER/PATH
    Options:
        --branch=BRANCH                 Specify the branch to use
        --revision=REVISION             Specify the revision to use
        --git-clone-option=OPTION       Pass OPTION to git clone
        --git-pull-option=OPTION        Pass OPTION to git pull (if --no-reset)
        --git-fetch-option=OPTION       Pass OPTION to git fetch (unless --no-reset)
        --no-reset                      Use git pull instead of git fetch and git reset --hard

git+file
    URL syntax: git+file:///PATH
    Options:
        --branch=BRANCH                 Specify the branch to use
        --revision=REVISION             Specify the revision to use
        --git-clone-option=OPTION       Pass OPTION to git clone
        --git-pull-option=OPTION        Pass OPTION to git pull (if --no-reset)
        --git-fetch-option=OPTION       Pass OPTION to git fetch (unless --no-reset)
        --no-reset                      Use git pull instead of git fetch and git reset --hard

git+http
    URL syntax: git+http://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --branch=BRANCH                 Specify the branch to use
        --revision=REVISION             Specify the revision to use
        --git-clone-option=OPTION       Pass OPTION to git clone
        --git-pull-option=OPTION        Pass OPTION to git pull (if --no-reset)
        --git-fetch-option=OPTION       Pass OPTION to git fetch (unless --no-reset)
        --no-reset                      Use git pull instead of git fetch and git reset --hard

git+https
    URL syntax: git+https://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --branch=BRANCH                 Specify the branch to use
        --revision=REVISION             Specify the revision to use
        --git-clone-option=OPTION       Pass OPTION to git clone
        --git-pull-option=OPTION        Pass OPTION to git pull (if --no-reset)
        --git-fetch-option=OPTION       Pass OPTION to git fetch (unless --no-reset)
        --no-reset                      Use git pull instead of git fetch and git reset --hard

git+rsync
    URL syntax: git+rsync://[USERNAME@]SERVER[:PORT]/PATH
    Options:
        --branch=BRANCH                 Specify the branch to use
        --revision=REVISION             Specify the revision to use
        --git-clone-option=OPTION       Pass OPTION to git clone
        --git-pull-option=OPTION        Pass OPTION to git pull (if --no-reset)
        --git-fetch-option=OPTION       Pass OPTION to git fetch (unless --no-reset)
        --no-reset                      Use git pull instead of git fetch and git reset --hard

git+ssh
    URL syntax: git+ssh://[USERNAME[:PASSWORD]@]SERVER/PATH
    Options:
        --branch=BRANCH                 Specify the branch to use
        --revision=REVISION             Specify the revision to use
        --git-clone-option=OPTION       Pass OPTION to git clone
        --git-pull-option=OPTION        Pass OPTION to git pull (if --no-reset)
        --git-fetch-option=OPTION       Pass OPTION to git fetch (unless --no-reset)
        --no-reset                      Use git pull instead of git fetch and git reset --hard

hg+file
    URL syntax: hg+file:///PATH[#BRANCH]
    Options:
        --hg-option=OPTION              Pass OPTION to hg as a global option
        --hg-clone-option=OPTION        Pass OPTION to hg clone
        --hg-pull-option=OPTION         Pass OPTION to hg pull
        --hg-update-option=OPTION       Pass OPTION to hg update

hg+http
    URL syntax: hg+http://[USERNAME@]SERVER/PATH[#BRANCH]
    Options:
        --hg-option=OPTION              Pass OPTION to hg as a global option
        --hg-clone-option=OPTION        Pass OPTION to hg clone
        --hg-pull-option=OPTION         Pass OPTION to hg pull
        --hg-update-option=OPTION       Pass OPTION to hg update

hg+https
    URL syntax: hg+https://[USERNAME@]SERVER/PATH[#BRANCH]
    Options:
        --hg-option=OPTION              Pass OPTION to hg as a global option
        --hg-clone-option=OPTION        Pass OPTION to hg clone
        --hg-pull-option=OPTION         Pass OPTION to hg pull
        --hg-update-option=OPTION       Pass OPTION to hg update

hg+ssh
    URL syntax: hg+ssh://[USERNAME@]SERVER[:PORT]/PATH[#BRANCH]
    Options:
        --hg-option=OPTION              Pass OPTION to hg as a global option
        --hg-clone-option=OPTION        Pass OPTION to hg clone
        --hg-pull-option=OPTION         Pass OPTION to hg pull
        --hg-update-option=OPTION       Pass OPTION to hg update

hg+static-http
    URL syntax: hg+static-http://SERVER[:PORT]/PATH[#BRANCH]
    Options:
        --hg-option=OPTION              Pass OPTION to hg as a global option
        --hg-clone-option=OPTION        Pass OPTION to hg clone
        --hg-pull-option=OPTION         Pass OPTION to hg pull
        --hg-update-option=OPTION       Pass OPTION to hg update

rsync
    URL syntax: rsync://[USERNAME@]SERVER[:PORT]/PATH
    Options:
        --exclude=PATTERN               Use PATTERN as an exclude pattern
        --exclude-from=FILE             Use FILE as a list of exclude patterns
        --rsync-option=OPTION           Pass OPTION to rsync

rsync+ssh
    URL syntax: rsync+ssh://[USERNAME@]SERVER:/PATH
    Options:
        --exclude=PATTERN               Use PATTERN as an exclude pattern
        --exclude-from=FILE             Use FILE as a list of exclude patterns
        --rsync-option=OPTION           Pass OPTION to rsync

svn
    URL syntax: svn://[USERNAME@]SERVER/PATH
    Options:
        --svn-checkout-option=OPTION    Pass OPTION to svn checkout

svn+file
    URL syntax: svn+file:///PATH
    Options:
        --svn-checkout-option=OPTION    Pass OPTION to svn checkout

svn+http
    URL syntax: svn+http://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --svn-checkout-option=OPTION    Pass OPTION to svn checkout

svn+https
    URL syntax: svn+https://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --svn-checkout-option=OPTION    Pass OPTION to svn checkout

svn+ssh
    URL syntax: svn+ssh://[USERNAME[:PASSWORD]@]SERVER:/PATH
    Options:
        --svn-checkout-option=OPTION    Pass OPTION to svn checkout

tar+file
    URL syntax: tar+file:///PATH
    Options:
        --exclude=PATTERN               Use PATTERN as an exclude pattern
        --exclude-from=FILE             Use FILE as a list of exclude patterns
        --strip-components=N            Ignore the first N components of the
                                          paths from the tarball
        --tmpdir=DIR                    Use DIR as base directory for
                                          temporary files (/var/tmp by default)
        --fetcher-option=OPTION         Pass OPTION to the fetcher
                                          (The standard fetchers do not
                                           accept any options - this is only
                                           useful with custom fetchers)
        --rsync-option=OPTION           Pass OPTION to the local rsync
        --tar-option=OPTION             Pass OPTION to tar

tar+ftp
    URL syntax: tar+ftp://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --exclude=PATTERN               Use PATTERN as an exclude pattern
        --exclude-from=FILE             Use FILE as a list of exclude patterns
        --strip-components=N            Ignore the first N components of the
                                          paths from the tarball
        --tmpdir=DIR                    Use DIR as base directory for
                                          temporary files (/var/tmp by default)
        --fetcher-option=OPTION         Pass OPTION to the fetcher
                                          (The standard fetchers do not
                                           accept any options - this is only
                                           useful with custom fetchers)
        --rsync-option=OPTION           Pass OPTION to the local rsync
        --tar-option=OPTION             Pass OPTION to tar

tar+http
    URL syntax: tar+http://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --exclude=PATTERN               Use PATTERN as an exclude pattern
        --exclude-from=FILE             Use FILE as a list of exclude patterns
        --strip-components=N            Ignore the first N components of the
                                          paths from the tarball
        --tmpdir=DIR                    Use DIR as base directory for
                                          temporary files (/var/tmp by default)
        --fetcher-option=OPTION         Pass OPTION to the fetcher
                                          (The standard fetchers do not
                                           accept any options - this is only
                                           useful with custom fetchers)
        --rsync-option=OPTION           Pass OPTION to the local rsync
        --tar-option=OPTION             Pass OPTION to tar

tar+https
    URL syntax: tar+https://[USERNAME[:PASSWORD]@]SERVER[:PORT]/PATH
    Options:
        --exclude=PATTERN               Use PATTERN as an exclude pattern
        --exclude-from=FILE             Use FILE as a list of exclude patterns
        --strip-components=N            Ignore the first N components of the
                                          paths from the tarball
        --tmpdir=DIR                    Use DIR as base directory for
                                          temporary files (/var/tmp by default)
        --fetcher-option=OPTION         Pass OPTION to the fetcher
                                          (The standard fetchers do not
                                           accept any options - this is only
                                           useful with custom fetchers)
        --rsync-option=OPTION           Pass OPTION to the local rsync
        --tar-option=OPTION             Pass OPTION to tar

User Defined Syncers

To write your own syncer for protocol proto, create an executable script named doproto (the do prefix is essential). Syncers may be located in any of the following directories:

confdir/syncers/, where confdir is the directory in which use.conf et al. reside.
DATADIR/paludis/syncers/. On most systems, DATADIR is /usr/share.
LIBEXECDIR/paludis/syncers/. On most systems, LIBEXECDIR is /usr/libexec.

Paludis places its own syncers in LIBEXECDIR/syncers/doproto. This directory is not for end user use.

A syncer script can be called in two ways:

With --help as an argument. When this happens, it must output a useful help message and then exit without syncing anything.
With two arguments, the first being the local directory and the second being the remote sync URL. If sync_options is used, its contents are also passed as arguments. Success or failure is indicated via exit status.

Many syncers use source "${PALUDIS_EBUILD_DIR}/echo_functions.bash" early on to get access to ewarn and friends.

For examples, consult the built-in syncers, which can be found in LIBEXECDIR/paludis/syncers/.

Introduction	Hooks	general.conf	bashrc
Bugs, Requests, Support	Syncers	keywords.conf / platforms.conf	suggestions.conf
Overview	Fetchers	use.conf / options.conf	sets/
FAQ	Env Vars	licenses.conf / licences.conf	repositories/
Clients	File Formats	mirrors.conf
Configuration	Dep Specs	output.conf
API	specpath.conf	package_mask.conf