go_modules: how to create go vendor tarballs from subdirectories

openSUSE

The go_modules OBS service is used to download, verify, and vendor Go module dependency sources.

As described in the source project’s (obs-service-go_modules) README:

Using the go.mod and go.sum files present in a Go application, obs-service-go_modules will call Go tools in sequence:

• go mod download
• go mod verify
• go mod vendor

obs-service-go_modules then creates a vendor.tar.gz archive (or another supported compression format) containing the vendor/ directory generated by go mod vendor.
This archive is produced in the RPM package directory and can be committed to OBS to support offline Go application builds for openSUSE, SUSE, and various other distributions.

The README also provides a few usage examples for packagers.

However, it wasn’t immediately clear how to use the go_modules OBS service to create multiple vendor tarballs from different subdirectories within a single Git source repository.

Below is an example where I create multiple vendor tarballs from a single Git repo— (in this case, the etcd project):

<services>

  <!-- Service #1 -->
  <service name="obs_scm">
    <param name="url">https://github.com/etcd/etcd.git</param>
    <param name="scm">git</param>
    <param name="package-meta">yes</param>
    <param name="versionformat">@PARENT_TAG@</param>
    <param name="versionrewrite-pattern">v(.*)</param>
    <param name="revision">v3.5.21</param>
    <param name="without-version">yes</param>
  </service>

  <!-- Service #2 -->
  <service name="go_modules">
    <param name="archive">*etcd.obscpio</param>
  </service>

  <!-- Service #3 -->
  <service name="go_modules">
    <param name="archive">*etcd.obscpio</param>
    <param name="subdir">server</param>
    <param name="vendorname">vendor-server</param>
  </service>

  <!-- Service #4 -->
  <service name="go_modules">
    <param name="archive">*etcd.obscpio</param>
    <param name="subdir">etcdctl</param>
    <param name="vendorname">vendor-etcdctl</param>
  </service>

</services>

The above _service file defines four services:

• Service 1 clones the GitHub repo github.com/etcd/etcd.git into the build root. The resulting output is a cpio archive blob—etcd.cpio.

• Service 2 locates the etcd.cpio archive, extracts it, runs go mod download, go mod verify, and go mod vendor from the repo root, and creates the default vendor.tar.gz.

• Service 3 and Service 4 work the same as Service 2, with one difference: they run the Go module commands from subdirectories:

  • Service 3 changes into the server/ directory before running the Go commands, producing a tarball named vendor-server.tar.gz.
  • Service 4 does the same for the etcdctl/ directory, producing vendor-etcdctl.tar.gz.

🔍 Note the subdir and vendorname parameters. These are the key to generating multiple vendor tarballs from various subdirectories, with custom names.

I found the full list of parameters accepted by the go_modules service defined here¹:

...
parser.add_argument("--strategy", default="vendor")
parser.add_argument("--archive")
parser.add_argument("--outdir")
parser.add_argument("--compression", default=DEFAULT_COMPRESSION)
parser.add_argument("--basename")
parser.add_argument("--vendorname", default=DEFAULT_VENDOR_STEM)
parser.add_argument("--subdir")
...

The default values are defined here²:

DEFAULT_COMPRESSION = "gz"
DEFAULT_VENDOR_STEM = "vendor"

Also, while writing this post, I discovered that the final vendor tarball can be compressed in one of the following supported formats³:

.tar.bz2
.tar.gz
.tar.lz
.tar.xz
.tar.zst

And finally, here’s the list of supported source archive formats (the blob from which the vendor tarball is created), powered by the libarchive Python module⁴:

READ_FORMATS = set((
    '7zip', 'all', 'ar', 'cab', 'cpio', 'empty', 'iso9660', 'lha', 'mtree',
    'rar', 'raw', 'tar', 'xar', 'zip', 'warc'
))