Snapcraft

stage-snaps don't work on docker

Bug #1885028 reported by Kyle Fazzari on 2020-06-24

This bug affects 3 people

Affects		Status	Importance	Assigned to	Milestone
	Snapcraft	Triaged	Low	Sergio Schvezov

Bug Description

Take this part:

parts:
  foo:
    plugin: nil
    stage-snaps: [go]

It doesn't work in docker, failing with:

# snapcraft
The following snaps are required but not installed as snapcraft is running inside docker or podman container: core18.
Please ensure the environment is properly setup before continuing.
Ignore this message if the appropriate measures have already been taken
Sorry, an error occurred in Snapcraft:
('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

If I look at the traceback, it looks like this:

# cat /tmp/tmpkhsge2v6/trace.txt
Traceback (most recent call last):
  File "/snap/snapcraft/current/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/snap/snapcraft/current/lib/python3.6/site-packages/requests_unixsocket/adapters.py", line 32, in connect
    sock.connect(socket_path)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/snapcraft/current/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/snap/snapcraft/current/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/snap/snapcraft/current/lib/python3.6/site-packages/urllib3/util/retry.py", line 368, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/snap/snapcraft/current/usr/lib/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/snap/snapcraft/current/lib/python3.6/site-packages/requests_unixsocket/adapters.py", line 32, in connect
    sock.connect(socket_path)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/snapcraft/current/bin/snapcraft", line 11, in <module>
    load_entry_point('snapcraft==3.9.1', 'console_scripts', 'snapcraft')()
  File "/snap/snapcraft/current/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/click/core.py", line 1236, in invoke
    return Command.invoke(self, ctx)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/cli/_runner.py", line 104, in run
    snap_command.invoke(ctx)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/cli/_command.py", line 88, in invoke
    return super().invoke(ctx)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/cli/lifecycle.py", line 280, in snap
    _execute(steps.PRIME, parts=[], pack_project=True, output=output, **kwargs)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/cli/lifecycle.py", line 78, in _execute
    lifecycle.execute(step, project_config, parts)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 131, in execute
    executor.run(step, part_names)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 185, in run
    self._handle_step(part_names, part, step, current_step, cli_config)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 199, in _handle_step
    getattr(self, "_run_{}".format(current_step.name))(part)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 241, in _run_pull
    self._run_step(step=steps.PULL, part=part, progress="Pulling")
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 315, in _run_step
    self._prepare_step(step=step, part=part)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 307, in _prepare_step
    preparation_function()
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 450, in prepare_pull
    self._fetch_stage_snaps()
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 412, in _fetch_stage_snaps
    snaps_list=stage_snaps, directory=self.plugin.snapsdir
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/repo/snaps.py", line 278, in download_snaps
    if not snap_pkg.is_valid():
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/repo/snaps.py", line 168, in is_valid
    if not self.in_store:
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/repo/snaps.py", line 91, in in_store
    self._is_in_store = self.get_store_snap_info() is not None
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/repo/snaps.py", line 114, in get_store_snap_info
    self._store_snap_info = _get_store_snap_info(self.name)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/snapcraft/internal/repo/snaps.py", line 390, in _get_store_snap_info
    snap_info = requests_unixsocket.get(url)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/requests_unixsocket/__init__.py", line 51, in get
    return request('get', url, **kwargs)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/requests_unixsocket/__init__.py", line 46, in request
    return session.request(method=method, url=url, **kwargs)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/requests/sessions.py", line 524, in request
    resp = self.send(prep, **send_kwargs)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/requests/sessions.py", line 637, in send
    r = adapter.send(request, **kwargs)
  File "/snap/snapcraft/current/lib/python3.6/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

It makes sense why build-snaps aren't supported in docker, but I feel like stage-snaps should work.

Revision history for this message

Kyle Fazzari (kyrofa) wrote on 2020-06-25:

From IRC, it seems this is due to snapcraft relying on the `snap` command to download snaps, which therefore depends upon snapd, which means stage-snaps don't work for the same reason build-snaps don't work: snapd doesn't work in docker.

However, it doesn't need to be that way. stage-snaps SHOULDN'T rely on snapd working-- they're just downloaded and unpacked, which is something snapcraft could definitely do on its own. Thoughts?

Revision history for this message

Tony Espy (awe) wrote on 2020-07-13:

What version of snapcraft was this bug introduced? Our EdgeX LF CI builds are now failing due to this.

Revision history for this message

Ian Johnson (anonymouse67) wrote on 2020-07-13:

@awe this is a different bug than we are seeing FWIW, the bug we are seeing is that now snapcraft requires the snap command to be present in order to call `snap pack`.

Revision history for this message

Kyle Fazzari (kyrofa) wrote on 2020-07-16:

Ian, I think the fix is the same: what if the snap command was shipped as snapd-client or similar? That way snapcraft wouldn't depend on snapd?

Revision history for this message

Sergio Schvezov (sergiusens) wrote on 2020-07-16: Re: [Bug 1885028] Re: stage-snaps don't work on docker

Kyle Fazzari <email address hidden> writes:

> Ian, I think the fix is the same: what if the snap command was
> shipped
> as snapd-client or similar? That way snapcraft wouldn't depend
> on snapd?

We download through snapd to take advantage of "snapd" as a cache
in some
scenarios and for "advanced grammar" validity checks. We could
fallback to using
the Snap Store if "snap download" somehow requires snapd to be
running.

But if that fallback is in place, we will need to fail advanced
grammar with a
caveat of that functionality being not supported under docker for
snaps.

Sergio Schvezov (sergiusens) on 2021-03-25

Changed in snapcraft:
importance:	Undecided → Low
status:	New → Triaged
assignee:	nobody → Sergio Schvezov (sergiusens)

Revision history for this message

Anton Alexandrenok (the-spyke) wrote on 2021-10-27:

I have the same issue following the official CircleCI tutorial: https://forum.snapcraft.io/t/circleci-tutorial-doesnt-work/27263

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.