We are Blocked by CDN

[lenliscio]

We seem to be blocked from downloading the virus definitions.
We have stopped all our servers from running freshclam.
We wait 24 hours, many times, and it tells again and again that we have to wait and that we are blocked.
Below is the Freshclam output, and the clamconf -n below.
I deleted the freshclam.dat file before running again this morning.
We were at an older version, and have just upgraded to 1.3.1.

rpm -qa | grep clamav

clamav-1.3.1-1.x86_64

freshclam -vvv

Connecting via proxysg.symcor.com
Current working dir is /var/clamav/
Can't open freshclam.dat in /var/clamav
It probably doesn't exist yet. That's ok.
Failed to load freshclam.dat; will create a new freshclam.dat
Creating new freshclam.dat
Saved freshclam.dat
ClamAV update process started at Mon Jun 24 09:21:52 2024
Current working dir is /var/clamav/
Querying current.cvd.clamav.net
TTL: 549
fc_dns_query_update_info: Software version from DNS: 0.103.11
Current working dir is /var/clamav/
check_for_new_database_version: Local copy of daily found: daily.cvd.
query_remote_database_version: daily.cvd version from DNS: 27316
daily database available for update (local version: 27285, remote version: 27316)
Retrieving https://database.clamav.net/daily.cvd
Using proxy: proxysg.symcor.com:80
downloadFile: Download source:      https://database.clamav.net/daily.cvd
downloadFile: Download destination: /var/clamav/tmp.fc6487a140/clamav-4e8f107ab40a1f17bfef20a218d4e693.tmp
* Host proxysg.symcor.com:80 was resolved.
* IPv6: (none)
* IPv4: 172.25.242.36
*   Trying 172.25.242.36:80...
* Connected to proxysg.symcor.com (172.25.242.36) port 80
* CONNECT tunnel: HTTP/1.1 negotiated
* allocate connect buffer
* Establish HTTP proxy tunnel to database.clamav.net:443
> CONNECT database.clamav.net:443 HTTP/1.1
Host: database.clamav.net:443
User-Agent: ClamAV/1.3.1 (OS: Linux, ARCH: x86_64, CPU: x86_64, UUID: 5963439d-4c8e-4134-83b9-b0e35cbbed5a)
Proxy-Connection: Keep-Alive

< HTTP/1.1 200 Connection established
<
* CONNECT phase completed
* CONNECT tunnel established, response 200
* ALPN: curl offers h2,http/1.1
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: /etc/ssl/certs
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / RSASSA-PSS
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=database.clamav.net
*  start date: Jun 12 03:53:03 2024 GMT
*  expire date: Sep 10 03:53:02 2024 GMT
*  subjectAltName: host "database.clamav.net" matched cert's "database.clamav.net"
*  issuer: C=ca; CN=symcor_proxysg
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/1.x
> GET /daily.cvd HTTP/1.1
Host: database.clamav.net
User-Agent: ClamAV/1.3.1 (OS: Linux, ARCH: x86_64, CPU: x86_64, UUID: 5963439d-4c8e-4134-83b9-b0e35cbbed5a)
Accept: */*
If-Modified-Since: Fri, 24 May 2024 08:30:55 GMT
Connection: close

* Request completely sent off
< HTTP/1.1 403 Forbidden
< Cache-Control: no-cache
< X-XSS-Protection: 1
< Connection: Keep-Alive
< Content-Type: text/html; charset=utf-8
< Content-Length: 7340
< Pragma: no-cache
<
Time:    0.2s, ETA:    0.0s [========================>]    7.17KiB/7.17KiB
* Connection #0 to host proxysg.symcor.com left intact
Saved freshclam.dat
WARNING: Can't download daily.cvd from https://database.clamav.net/daily.cvd
WARNING: FreshClam received error code 403 from the ClamAV Content Delivery Network (CDN).
This could mean several things:
 1. You are running an out-of-date version of ClamAV / FreshClam.
    Ensure you are the most updated version by visiting https://www.clamav.net/downloads
 2. Your network is explicitly denied by the FreshClam CDN.
    In order to rectify this please check that you are:
   a. Running an up-to-date version of FreshClam
   b. Running FreshClam no more than once an hour
   c. If you have checked (a) and (b), please open a ticket at
      https://github.com/Cisco-Talos/clamav/issues
      and we will investigate why your network is blocked.
WARNING: You are on cool-down until after: 2024-06-25 09:21:53
ERROR: Database update process failed: Forbidden; Blocked by CDN
ERROR: Update failed.

clamconf -n

Checking configuration files in /usr/local/etc

Config file: clamd.conf
-----------------------
LogFile = "/var/log/clamav/clamd.log"
LogTime = "yes"
LogClean = "yes"
LogSyslog = "yes"
LogFacility = "LOG_MAIL"
LogVerbose = "yes"
LogRotate = "yes"
DatabaseDirectory = "/var/clamav"
LocalSocket = "/tmp/clamd.socket"
LocalSocketMode = "660"
ExcludePath = "^/proc/", "^/sys/", "^/var/log/"

Config file: freshclam.conf
---------------------------
LogSyslog = "yes"
DatabaseDirectory = "/var/clamav"
UpdateLogFile = "/var/log/clamav/freshclam.log"
DatabaseMirror = "db.ca.clamav.net", "db.local.clamav.net"
ScriptedUpdates disabled
HTTPProxyServer = "proxysg.symcor.com"
HTTPProxyPort = "80"

clamav-milter.conf not found

Software settings
-----------------
Version: 1.3.1
Optional features supported: MEMPOOL AUTOIT_EA06 BZIP2 LIBXML2 PCRE2 ICONV JSON RAR

Database information
--------------------
Database directory: /var/clamav
bytecode.cvd: version 335, sigs: 86, built on Tue Feb 27 10:37:24 2024
daily.cvd: version 27285, sigs: 2061644, built on Fri May 24 04:30:55 2024
main.cvd: version 62, sigs: 6647427, built on Thu Sep 16 08:32:42 2021
Total number of signatures: 8709157

Platform information
--------------------
uname: Linux 4.18.0-553.5.1.el8_10.x86_64 #1 SMP Tue May 21 03:13:04 EDT 2024 x86_64
OS: Linux, ARCH: x86_64, CPU: x86_64
Full OS version: "Red Hat Enterprise Linux release 8.10 (Ootpa)"
zlib version: 1.3.1 (1.3.1), compile flags: a9
platform id: 0x0a21c9c90800000002040805

Build information
-----------------
GNU C: 4.8.5 20150623 (Red Hat 4.8.5-44) (4.8.5)
sizeof(void*) = 8
Engine flevel: 201, dconf: 201

[micahsnyder]

@lenliscio The most like cases for a 403 response are:

1. Being affected by geolocation IP blocks the result of US sanctions (i.e. see ClamAV site and update database blocked by CDN in Russia #500).
2. Using a program other than Freshclam or CVDUpdate to update. We use HTTP User-Agent header checks to enforce this. It helps reduce traffic from those who would do full database downloads more than once a day, for no good reason. We only update the database once a day and Freshclam has the ability to update using small patch files. These are easily foiled, I admit, and we only ask that our users be respectful.
3. Using an outdated ClamAV version that has been intentionally blocked. See the "DB downloads allowed until" column in our EOL matrix.

I suspect that your issue has to do with your proxy configuration. My guess is that it isn't forwarding Freshclam's User-Agent and so is being blocked.

[lenliscio]

Hi, Has anything changed the way freshclam works with these newer versions? We've had our proxy working with freshclam for 10 years now, and only recently it has stopped working. With the old version we had, which I suspect the version was too old, and now with the newer version.

[micahsnyder]

Freshclam changed significantly in ClamAV 0.102.0, but it has not changed much since.

[lenliscio]

HI, is it possible we are blocked on the server side where we download the definitions from? From cloudflare?

[micahsnyder]

If you can gather the cloudflare cf-ray ID number, we can look for the log entry in our cloudflare account dashboard. If we have a log entry, it should tell us why you're being blocked. If we don't have a log entry, then it may mean that cloudflare itself has blocked you.

To do this, first delete the freshclam.dat file found in your database directory. Then run "freshclam --verbose" like you did in the report. In the response from Cloudflare, you should see something along the lines of:

< cf-ray: 874a2fa8af2c9295-FRA

Gather this and share it here.

If you can share your public IP address, that may also help -- though the cf-ray ID is most important to find the specific event log entry.

[lenliscio]

I found possible of 2 ip addresses we have.
205.189.242.100
205.189.240.2

We do not see any cf-ray ID. I actually pasted the full output above, and there isn't a cf-ray anywhere in there.

[micahsnyder]

@lenliscio I spoke with our cloudflare account admin. They looked up log entries as found the following:

Not all of their requests 403, the only 403's for these IPs are requests to daily.cvd and bytecode.cvd, which are the paths where HTTP User-Agent is enforced.
Some requests come from Edge, and some from Chrome, but those paths support neither. If they can fix their proxy, they should be OK.

You must not use a browser to download the signature databases. You must use Freshclam or CVDUpdate. The response I got from our cloudflare admin confirms my guess is that your proyx isn't forwarding Freshclam's User-Agent HTTP header and so is being blocked. Please worth with your proxy administrator to try to solve the issue.