When writing the haproxy.cfg file, a `TypeError: write() argument must be str, not bytes` is thrown.

Bug #2033066 reported by Marcelo Subtil Marcal
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
charm-haproxy
Fix Released
Undecided
Unassigned

Bug Description

When deploying haproxy, channel `latest/stable` revision `74`, the unit gets stuck into an error state:

```
Unit Workload Agent Machine Public address Ports Message
landscape-server-haproxy/0* error idle 2 10.243.175.32 hook failed: "reverseproxy-relation-changed"
  filebeat/0* active idle 10.243.175.32 Filebeat ready.
  landscape-client/0* maintenance idle 10.243.175.32 Need computer-title and juju-info to proceed
  logrotated/0* active idle 10.243.175.32 Unit is ready.
  nrpe/0* active idle 10.243.175.32 icmp,5666/tcp Ready
  ntp/0* active idle 10.243.175.32 123/udp chrony: Ready, OK: offset is -0.000037
  telegraf/0* active idle 10.243.175.32 9103/tcp Monitoring landscape-server-haproxy/0 (source version/commit 23.07)
  ubuntu-advantage/0* active idle 10.243.175.32 Attached (esm-apps,esm-infra,livepatch)
  ```

The unit log shows:

```
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 Traceback (most recent call last):
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 File "/var/lib/juju/agents/unit-landscape-server-haproxy-0/charm/hooks/reverseproxy-relation-changed", line 1575, in <module>
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 main(hook_name)
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 File "/var/lib/juju/agents/unit-landscape-server-haproxy-0/charm/hooks/reverseproxy-relation-changed", line 1542, in main
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 reverseproxy_interface("changed")
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 File "/var/lib/juju/agents/unit-landscape-server-haproxy-0/charm/hooks/reverseproxy-relation-changed", line 1095, in reverseproxy_interface
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 config_changed()
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 File "/var/lib/juju/agents/unit-landscape-server-haproxy-0/charm/hooks/reverseproxy-relation-changed", line 1032, in config_changed
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 if not create_services():
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 File "/var/lib/juju/agents/unit-landscape-server-haproxy-0/charm/hooks/reverseproxy-relation-changed", line 754, in create_services
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 write_service_config(services_dict)
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 File "/var/lib/juju/agents/unit-landscape-server-haproxy-0/charm/hooks/reverseproxy-relation-changed", line 836, in write_service_config
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 f.write(base64.b64decode(errorfile["content"]))
2023-08-25 12:25:18 WARNING unit.landscape-server-haproxy/0.reverseproxy-relation-changed logger.go:60 TypeError: write() argument must be str, not bytes
2023-08-25 12:25:18 ERROR juju.worker.uniter.operation runhook.go:153 hook "reverseproxy-relation-changed" (via explicit, bespoke hook script) failed: exit status 1
```

The charm configuration is:
```
  landscape-server-haproxy:
    bindings:
      ? ''
      : oam-space
    charm: haproxy
    num_units: 1
    options:
      default_timeouts: queue 60000, connect 5000, client 120000, server 120000
      global_default_bind_options: no-tlsv10
      services: ''
      ssl_cert: include-base64://../../tls/landscape/landscape.crt
      ssl_key: include-base64://../../tls/landscape/landscape.key
    to:
    - '14'
```

The last known working charm revision was `66`.

Related branches

Revision history for this message
Marcelo Subtil Marcal (msmarcal) wrote :

Subscribed ~field-critical since this is blocking the cloud handover and the last known charm revision is not available on charmstore

Revision history for this message
Marcelo Subtil Marcal (msmarcal) wrote :

There is a workaround: deploying the haproxy revision 66.

That could be done adding this to an overlay:
```
  landscape-server-proxy:
    charm: haproxy
    channel: stable
    revision: 66
```

Removing ~field-critical and subscribing ~field-high

Revision history for this message
Tom Haddon (mthaddon) wrote :

I believe this is the fix, but I'm still trying to reproduce the bug:

```
$ bzr di
=== modified file 'hooks/hooks.py'
--- old/hooks/hooks.py 2023-08-09 07:20:52 +0000
+++ new/hooks/hooks.py 2023-08-27 13:13:08 +0000
@@ -833,7 +833,7 @@
             full_path = os.path.join(
                 path, "%s.http" % errorfile["http_status"])
             with open(full_path, 'w') as f:
- f.write(base64.b64decode(errorfile["content"]))
+ f.write(base64.b64decode(errorfile["content"]).decode('utf-8'))

         # Write to disk the content of the given SSL certificates
         crts = service_config.get('crts', [])

```

I've tried recreating the issue with the following bundle, but have so far been unable to do so:
```
# Create certs as follows:
# openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 3650 -nodes -subj "/C=XX/ST=StateName/L=CityName/O=CompanyName/OU=CompanySectionName/CN=CommonNameOrHostname"
#
applications:
  haproxy:
    charm: haproxy
    num_units: 1
    series: jammy
    options:
      default_timeouts: queue 60000, connect 5000, client 120000, server 120000
      global_default_bind_options: no-tlsv10
      services: ''
      ssl_cert: include-base64://cert.pem
      ssl_key: include-base64://key.pem
  pollen:
    charm: pollen
    channel: latest/edge
    series: jammy
    num_units: 1
relations:
  - - haproxy
    - pollen
```
Can you confirm if the options mentioned in the bug report are the full list of options to reproduce the bug? Or can you provide a reproducer?

Revision history for this message
Marcelo Subtil Marcal (msmarcal) wrote :

Hello Tom,

Yes, those are the options used for the deployment.

The script used for generating the certs is:
```
#!/bin/bash

# Directory to store certificates
CERTIFICATES_DIR=tls/landscape

# Validity duration in days
DURATION=3560

SUBJECT="/C=BR/ST=Sao Paulo/L=Sao Paulo/O=Customer/CN=landscapeha-1.maas"

mkdir -p ${CERTIFICATES_DIR}

# Generate landscape self-signed certificate
openssl req -x509 -newkey rsa:4096 -nodes \
  -keyout ${CERTIFICATES_DIR}/landscape.key \
  -out ${CERTIFICATES_DIR}/landscape.crt \
  -subj "${SUBJECT}" \
  -days ${DURATION}

echo -n "base64:" > ${CERTIFICATES_DIR}/landscape.crt.b64
base64 ${CERTIFICATES_DIR}/landscape.crt >> ${CERTIFICATES_DIR}/landscape.crt.b64
```

Revision history for this message
Tom Haddon (mthaddon) wrote :

Ok, can you help with providing a reproducer? Which charm is related to this, can you give me a bundle that I can use to reproduce the issue?

Revision history for this message
Marcelo Subtil Marcal (msmarcal) wrote :

Hi Tom,

haproxy is related to landscape-server:

```
- ['landscape-server-haproxy:reverseproxy', 'landscape-server:website']
```

The full bundle: https://pastebin.canonical.com/p/TjmYPDTrbm/

Revision history for this message
Tom Haddon (mthaddon) wrote :

Ok, I've been able to reproduce the issue with the following bundle:
```
# Create certs as follows:
# openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 3650 -nodes -subj "/C=XX/ST=StateName/L=CityName/O=CompanyName/OU=CompanySectionName/CN=CommonNameOrHostname"
#
applications:
  haproxy:
    charm: haproxy
    num_units: 1
    series: jammy
    options:
      default_timeouts: queue 60000, connect 5000, client 120000, server 120000
      global_default_bind_options: no-tlsv10
      services: ''
      ssl_cert: include-base64://cert.pem
      ssl_key: include-base64://key.pem
  landscape-server:
    charm: landscape-server
    series: bionic
    num_units: 1
    options:
      ssl-cert: include-base64://cert.pem
      ssl-key: include-base64://key.pem
  postgresql:
    charm: postgresql
    channel: latest/stable
    series: jammy
    num_units: 1
relations:
  - - haproxy
    - landscape-server
  - - landscape-server:db
    - postgresql:db-admin
```
I've got a fix (not the one I was proposing), but am running into another problem, which I think is related to the version of HAProxy I'm using. I'll test again tomorrow with the same series as in the bundle above to confirm.

The fix I think will be:
```
$ bzr di
=== modified file 'hooks/hooks.py'
--- old/hooks/hooks.py 2023-08-09 07:20:52 +0000
+++ new/hooks/hooks.py 2023-08-29 16:30:45 +0000
@@ -832,7 +832,7 @@
             path = get_service_lib_path(service_name)
             full_path = os.path.join(
                 path, "%s.http" % errorfile["http_status"])
- with open(full_path, 'w') as f:
+ with open(full_path, 'wb') as f:
                 f.write(base64.b64decode(errorfile["content"]))

         # Write to disk the content of the given SSL certificates

```

Revision history for this message
Tom Haddon (mthaddon) wrote :

This has been fixed in revision 75

Changed in charm-haproxy:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.