[BUG] app_protocoland mode settings of long syntax in service ports section have no effect

[meyayl]

Description

Current behavior:

• configuration for app_protocol and mode are silently ignored and do not end up in created containers
• mode: host -> Docker 26.1.4: retains the remote ip, Docker < 26.0.0: uses bridge network gateway ip
• mode: ingress Docker 26.1.4 retains the remote ip, Docker < 26.0.0: uses bridge network gateway ip
• the implemented default behavior with Docker 26.1.4 does not match with the documentation, as the field is ignored, and behaves like mode: host. For Docker version < 26.0.0 the behavior matches the documentation

Expected behavior with Docker 26.1.4 and later:

• configuration for app_protocol and mode end up in created containers and have an effect, or at least inform the user they are not applied
• mode: host retains the source/remote ip
• mode: ingress does not retain the remote ip, but supports replicas
• the implemented default behavior aligns with the documentation (which says mode: ingress is default)

Steps To Reproduce

1. Create a compose file with different long syntax port settings:

$ cat compose.yml
services:
  whoami-retain-ip:
    image: "traefik/whoami"
    container_name: whoami-host
    ports:
    - name: web
      target: 80
      host_ip: 0.0.0.0
      published: "18888"
      protocol: tcp
      app_protocol: http
      mode: host
  whoami:
    image: "traefik/whoami"
    container_name: whoami
    ports:
    - 28888:80
  whoami-ingress:
    image: "traefik/whoami"
    container_name: whoami-ingress
    ports:
    - name: web
      target: 80
      host_ip: 0.0.0.0
      published: "38888"
      protocol: tcp
      app_protocol: http
      mode: ingress

2. Deploy the compose project with docker compose up -d
3. Inspect the network settings of each deployed container:

$ docker inspect --format '{{json .NetworkSettings.Ports}}' whoami-ingress  | jq
{
  "80/tcp": [
    {
      "HostIp": "0.0.0.0",
      "HostPort": "38888"
    }
  ]
}

$ docker inspect --format '{{json .NetworkSettings.Ports}}' whoami  | jq
{
  "80/tcp": [
    {
      "HostIp": "0.0.0.0",
      "HostPort": "28888"
    },
    {
      "HostIp": "::",
      "HostPort": "28888"
    }
  ]
}

$ docker inspect --format '{{json .NetworkSettings.Ports}}' whoami-host  | jq
{
  "80/tcp": [
    {
      "HostIp": "0.0.0.0",
      "HostPort": "18888"
    }
  ]
}

Compose Version

$ docker compose version
Docker Compose version v2.27.1

Docker Environment

$ docker info
Client: Docker Engine - Community
 Version:    26.1.4
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.1
    Path:     /home/me/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan
  scout: Docker Scout (Docker Inc.)
    Version:  v1.0.9
    Path:     /home/me/.docker/cli-plugins/docker-scout

Server:
 Containers: 15
  Running: 15
  Paused: 0
  Stopped: 0
 Images: 30
 Server Version: 26.1.4
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local pxd
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: {irrelevant}
  NodeID: {irrelevant}
  Is Manager: true
  ClusterID: {irrelevant}
  Managers: {irrelevant}
  Nodes: {irrelevant}
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 1
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: {irrelevant}
  Manager Addresses:
   {irrelevant}
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d2d58213f83a351ca8f528a95fbd145f5654e957
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
 Kernel Version: 5.4.0-182-generic
 Operating System: Ubuntu 20.04.6 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 11.68GiB
 Name: {irrelevant}
 ID: {irrelevant}
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: {irrelevant}
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support

Anything else?

No response

[ndeloof]

app_protocol is plain metadata and has no impact on docker engine

[ndeloof]

Need to check about mode. Maybe a change in Docker engine, or something missing in compose

[meyayl]

app_protocol is plain metadata and has no impact on docker engine

I am was uncertain, as the documentation is slightly ambiguous about it:

app_protocol: The application protocol (TCP/IP level 4 / OSI level 7) this port is used for. This is optional and can be used as a hint for Compose to offer richer behavior for protocols that it understands. Introduced in Docker Compose version 2.26.0.

What exactly does it do then? And what are the protocols it understands?

Regarding host:. it seems to be ignored, and as such we get the default behavior of the Docker Engine.

On that note,:the default behavior in Docker 26.x.x looks like what we know from mode: host for swarm services: the source ip is retained.

May I ask what the intended implementation for mode: ingress is? Should it behave like it does for swarm services and allow to loadbalance to replicas?

[ndeloof]

What exactly does it do then? And what are the protocols it understands?

This was introduced in the compose specification for parity with Kubernetes model and potential future improvements. This is unused for now in docker compose

About mode, AFAICT on a standalone docker engine this parameter is not relevant. This attribute was introduced in the compose file format by swarm, and is ignored in Docker Compose codebase

While the spec is confusing as such attributes comes from distinct sources and are not all implemented, what are you trying to achieve here ?

[meyayl]

My main objective is to prevent end user confusion :)

Back in the day when we had the v2 and v3 schema reference, I felt it helped a great deal that it pointed out when a particular feature was not available for swarm stack deployments, even though it was part of the schema version.

I kind of miss that in the current compose specification docs. I understand they refer to the specs themselves. I assume most users use docker compose v2. They find a documentation that is not on par with docker compose.

[ndeloof]

swarm does not use compose-specification, so you won't find any detail on supported feature unfortunately.
About supported features, we added tags in the spec (see https://github.com/compose-spec/compose-spec/blob/master/05-services.md#attach for example) to help user know minimal required version needed so they can rely on those.