Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[youtube] Sign in to confirm you’re not a bot. This helps protect our community #10128

Open
7 of 9 tasks
xpidev-afk opened this issue Jun 7, 2024 · 41 comments
Open
7 of 9 tasks
Labels
external issue Issue with an external tool site-bug Issue with a specific website

Comments

@xpidev-afk
Copy link

xpidev-afk commented Jun 7, 2024

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Please make sure the question is worded well enough to be understood

I've been having this problem for a day when I make a query from YouTube, it's more frequent than normal and I'm using proxies

ERROR: [youtube] 4TWR90KJl84: Sign in to confirm you’re not a bot. This helps protect our community. Learn more

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-Uv', '--proxy', 'xxxxx', 'Q0W6wcio384']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [ae2af1104] (zip)
[debug] Python 3.8.10 (CPython x86_64 64bit) - Linux-5.15.0-105-generic-x86_64-with-glibc2.29 (OpenSSL 1.1.1f  31 Mar 2020, glibc 2.31)
[debug] exe versions: ffmpeg 4.2.7, ffprobe 4.2.7
[debug] Optional libraries: sqlite3-3.31.1
[debug] Proxy map: {'all': 'xxxx'}
[debug] Request Handlers: urllib
[debug] Loaded 1820 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
ERROR: Unable to obtain version info (EOF occurred in violation of protocol (_ssl.c:1131)); Please try again later or visit  https://github.com/yt-dlp/yt-dlp/releases/latest
[youtube] Extracting URL: Q0W6wcio384
[youtube] Q0W6wcio384: Downloading webpage
[youtube] Q0W6wcio384: Downloading ios player API JSON
ERROR: [youtube] Q0W6wcio384: Sign in to confirm you’re not a bot. This helps protect our community. Learn more
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 734, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/youtube.py", line 4248, in _real_extract
    self.raise_no_formats(reason, expected=True)
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 1257, in raise_no_formats
    raise ExtractorError(msg, expected=expected, video_id=video_id)
@xpidev-afk xpidev-afk added the question Question label Jun 7, 2024
@TheSystemGuy1337
Copy link

TheSystemGuy1337 commented Jun 7, 2024

DO NOT PASS YOUR COOKIES, See #10085 and #10108.

@coletdjnz

This comment was marked as outdated.

@coletdjnz coletdjnz closed this as not planned Won't fix, can't repro, duplicate, stale Jun 7, 2024
@coletdjnz coletdjnz added the duplicate This issue or pull request already exists label Jun 7, 2024
@arstum

This comment was marked as spam.

@bashonly bashonly reopened this Jun 10, 2024
@bashonly bashonly changed the title Problem Sign in to confirm you’re not a bot youtube [youtube] Sign in to confirm you’re not a bot. This helps protect our community Jun 10, 2024
@bashonly bashonly added external issue Issue with an external tool site-bug Issue with a specific website and removed duplicate This issue or pull request already exists question Question labels Jun 10, 2024
This was referenced Jun 10, 2024
@bashonly
Copy link
Member

bashonly commented Jun 10, 2024

Reportedly, authenticating with oauth (via this plugin) instead of cookies has been working for some users.

This sign-in requirement is affecting almost all apps that access Youtube, including the official Youtube web app.

@formernest
Copy link

formernest commented Jun 11, 2024

i have this problem today, so what is do not pass cookie, it worked last week


EDIT BY MAINTAINER:
Yes, this is a new problem that didn't exist a ~week ago.
Read the linked issues to learn why passing cookies is not a good idea now.

@jakeogh
Copy link
Contributor

jakeogh commented Jun 11, 2024

@Foremost

Unless you were already using --cookies-from-browser no change is necessary. Youtube is blocking videos (for me at least) whether you are passing cookies from your web browser or not. By default yt-dlp does not pass cookies, and youtube may block your request anyway. Ultimately this arms race must* be on the JS execution level and what that JS engine presents to the user.

  • well, technically, we could all have opted out from executing arb code long ago... but the browsers were bought via direct funding to the controlling foundation (Firefox) and/or created (Chrome) by major content providers (ahum google).

@formernest
Copy link

i have this problem today, so what is do not pass cookie, it worked last week

EDIT BY MAINTAINER: Yes, this is a new problem that didn't exist a ~week ago. Read the linked issues to learn why passing cookies is not a good idea now.

I don't pass any cookies, when i use -vU to show debug log, but it still not worked. Is it a bug need fix?

@TheSystemGuy1337
Copy link

TheSystemGuy1337 commented Jun 11, 2024

I don't pass any cookies, when i use -vU to show debug log, but it still not worked. Is it a bug need fix?

YouTube has flagged and blocked your IP. You can try using OAuth to work around this, but there are no guarantees it will work for you. You may have to wait until you are assigned a new IP by your ISP. This can take around ~1 week to several months depending on your ISP. Most people have dynamic IPs, so your IP will eventually change. As I mentioned before, see #10085 and #10108 for more information. These issues are also listed in their FAQ.

@forxxin
Copy link

forxxin commented Jun 13, 2024

Embedding mode {'cookiesfrombrowser': ('firefox',),} works for now

cookiesfrombrowser: A tuple containing the name of the browser, the profile

cookiesfrombrowser:  A tuple containing the name of the browser, the profile
                   name/path from where cookies are loaded, the name of the keyring,
                   and the container name, e.g. ('chrome', ) or
                   ('vivaldi', 'default', 'BASICTEXT') or ('firefox', 'default', None, 'Meta')

@DimonKhutornoy
Copy link

DimonKhutornoy commented Jun 13, 2024

@forxxin , but i have next problem:
FileNotFoundError: could not find firefox cookies database in /home/$my name$/.mozilla/firefox.

Where i can find this file?

@joliedevil
Copy link

Your IP has been blacklisted by YouTube. Use a proxy with the following command:

yt-dlp --proxy "userProxy:passwordProxy@ipProxy" URL

@gamer191
Copy link
Collaborator

Can someone who's affected by this please send a verbose log of yt-dlp -v --extractor-args "youtube:player_client=tv_embedded,mediaconnect" URL

@TheSystemGuy1337
Copy link

Your IP has been blacklisted by YouTube. Use a proxy with the following command:

yt-dlp --proxy "userProxy:passwordProxy@ipProxy" URL

VPNs and proxies WILL NOT work as YouTube actively blocks them as well.

@coletdjnz
Copy link
Member

coletdjnz commented Jun 15, 2024

This is likely related to BotGuard where YouTube might be A/B testing/rolling some changes server-side. We are currently investigating this.

@NipLars
Copy link

NipLars commented Jun 16, 2024

Your IP has been blacklisted by YouTube. Use a proxy with the following command:
yt-dlp --proxy "userProxy:passwordProxy@ipProxy" URL

VPNs and proxies WILL NOT work as YouTube actively blocks them as well.

That is not so at all, I run vpn and works fine finding servers that has unblocked ip;s left.
Vpn is also circumventing those ISP doing CGNAT. CGNAT works a bit like vpn in the sense that same ip is shared by many.

This is my experience right now, having run vpn for 4 years.

Note! This goes for both for browsing and using yt-dlp

@jfeldhamer
Copy link

jfeldhamer commented Jun 16, 2024

seems yt-dl is working on my local machine but not on my contabo vps.
why wont my vps work? was a range of ips belonging to contabo all banned? i only downloaded about 100 videos a month maybe 200 tops.
should i setup a proxy to my local machine when the server wants to use yt-dlp to download? how do i setup a proxy on my window local machine that my vps can use?
or can i use nordvpn as a proxy?
is there a better recommendation?

@NipLars
Copy link

NipLars commented Jun 16, 2024

You cannot install vpn on a vps?

  • vps might be using CGNAT to share public ip:s and somebody flagged youtube
  • I don't know rules for Nordvpn, but there are those that allow install on three machines on a subscription.

@jfeldhamer
Copy link

You cannot install vpn on a vps?

  • vps might be using CGNAT to share public ip:s and somebody flagged youtube
  • I don't know rules for Nordvpn, but there are those that allow install on three machines on a subscription.

i was thinking using node as proxy i dont want to funnel all the vps traffic through nord just the youtube stuff. can i do that?

@NipLars
Copy link

NipLars commented Jun 16, 2024

If you have a lot of unattended things going on at vps, I might not have good advice for you.
But if attended, it's just connect vpn server or not while doing certain things.

There are split channel stuff that my email stuff go through without using vpn etc.

  • so particular processes can be set to use vpn or not

This might be to look for, think all vpn has this.

@devdanke
Copy link

Was getting similar error, "Get the best YouTube experience. Sign in to save videos..." and failed download.
Changed my VPN location and yt-dlp started working again.
I suspect this "fix" won't last forever.

@dadadies
Copy link

Also been having this issue the past week. Maybe its the way it is connecting and downloading from youtube and the method can be obfuscated so youtube doesn't know? Does this error occur when not using VPN - i cant test it as i need the VPN to have internet.

@caf2022
Copy link

caf2022 commented Jun 17, 2024

The problem is that youtube now is blocking the IP, then using a VPN or proxy work because is another IP... I use the yt-dlp on some linux servers and on the server that have much use the youtube has blocked, but I'm using a proxy at this moment. THe problem is that the free proxy don't have 100% uptime but at this moment is working. I can't beleave if this new block method can bypassed.

@wastemans
Copy link

This problem is entirely Google's fault for militarizing access control, to the final goal of the collection and warehousing of your data.
A change that was indeed introduced very recently.
If you use a VPN in conjunction with typical yt-dlp parameters, you're not contributing to their advertisers.
The simplest option is to switch VPN servers, or use a host that exposes a private, domestic IP.

@DimonKhutornoy
Copy link

Does anybody know how long the ban lasts? Maybe someone was unbanned in this time?

@formernest
Copy link

formernest commented Jun 20, 2024

As I know, it worked at 6.7, 6.10 have this problem. it still not work today

@TheSystemGuy1337
Copy link

TheSystemGuy1337 commented Jun 20, 2024

Does anybody know how long the ban lasts? Maybe someone was unbanned in this time?

We can assume the ban lasts around 72 hours (~3 days). Nobody has details about this particular ban, and no one has conducted research on how long the Sign in to confirm you're not a bot ban lasts. It could be 72 hours, or it could be permanent.

@terada46
Copy link

terada46 commented Jun 21, 2024

Thanks @caf2022 are you able to access YouTube without an account on the same IP address in your browser? Also, do you have any iOS devices you could test the YouTube app on? After testing that, can you please run yt-dlp -v --extractor-args "youtube:player_client=ios" URL to ensure you’re still “blocked”

I have this issue today. On my VPS server yt-dlp simply just don't work but as a proxy server I can access youtube on my browser through this IP address (haven't tested iOS device). I barely run yt-dlp on this IP address. Just once or twice a month. And before this issue occurred I ran yt-dlp to test if I can save live stream longer than 6 hours.

@terada46
Copy link

Thanks @caf2022 are you able to access YouTube without an account on the same IP address in your browser? Also, do you have any iOS devices you could test the YouTube app on? After testing that, can you please run yt-dlp -v --extractor-args "youtube:player_client=ios" URL to ensure you’re still “blocked”

I have this issue today. On my VPS server yt-dlp simply just don't work but as a proxy server I can access youtube on my browser through this IP address (haven't tested iOS device). I barely run yt-dlp on this IP address. Just once or twice a month. And before this issue occurred I ran yt-dlp to test if I can save live stream longer than 6 hours.

I can watch youtube on iOS device. And I tried to run yt-dlp -v --extractor-args "youtube:player_client=ios" URL. I'm still blocked.

@bashonly
Copy link
Member

Are you signed in on the iOS device?

@terada46
Copy link

Are you signed in on the iOS device?

Yes. The one I watch every day.

@bashonly
Copy link
Member

I don't see how being able to watch Youtube on the iOS device while signed in means anything? The issue is that Youtube is requiring sign-in

@Anachron
Copy link

Maybe I can clarify things:
I do not have a Youtube account anymore, but can Youtube just fine in the browser via a VPN and an IP that is known to come from such (cloudflare sometimes blocks my IP due to belonging to a VPS provider).

However, I do receive the same "Sign in to confirm you're not a bot." error message when trying to use yt-dlp on the command line.

So what I believe is happening here is that the web version sends some specific information to youtube to allow access to the stream data, which does not happen when trying to download it directly via the console.

I do not think this has anything to do with banning at all.

@TheSystemGuy1337
Copy link

Maybe I can clarify things: I do not have a Youtube account anymore, but can Youtube just fine in the browser via a VPN and an IP that is known to come from such (cloudflare sometimes blocks my IP due to belonging to a VPS provider).

However, I do receive the same "Sign in to confirm you're not a bot." error message when trying to use yt-dlp on the command line.

So what I believe is happening here is that the web version sends some specific information to youtube to allow access to the stream data, which does not happen when trying to download it directly via the console.

I do not think this has anything to do with banning at all.

There are cases where the Sign in to confirm you're not a bot error appears on official YouTube clients, including youtube.com

Source

@NipLars
Copy link

NipLars commented Jun 21, 2024

There's million things you can do with a browser, like run javascript to catch the full url.

  • Vimeo do it
  • Wistia do it
    Many business services rely on being able to control that only subscribers can stream, and absolutely not download and spread their lessons and whatnot.

So could be now that Google seen streamed traffic going down seriously, and ads too, they fine tune to really identify browser to they can stream

  • I imagine content providers also have an interest that everybody can stream, regardless signed in or not

I will test next time I get "sign in..." for yt-dlp if going to that url with browser also and see if that is rejected too.

EDIT: I did test after a reject from yt-dlp and then with Brave browser, no difference.

So what User-Agent is the browser that work have that always works?

Mine looks like this "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36" can also be taken from yt-dlp with parameter
" --dump-user-agent Display the current user-agent and exit
"

Somebody mentioned Firefox and some no cookies parameter that worked?

Only logic I found is ip-banning....

@lolcatw
Copy link

lolcatw commented Jun 22, 2024

seems yt-dl is working on my local machine but not on my contabo vps. why wont my vps work? was a range of ips belonging to contabo all banned? i only downloaded about 100 videos a month maybe 200 tops. should i setup a proxy to my local machine when the server wants to use yt-dlp to download? how do i setup a proxy on my window local machine that my vps can use? or can i use nordvpn as a proxy? is there a better recommendation?

I have the same issue with the same provider. The answer is to get yourself some residential proxies to work around the issue. @jfeldhamer

@accessiblepixel
Copy link

I've had my residential IP blocked by YouTube for two weeks now, and it won't let any of us watch it without being signed in - I used an invidious instance locally for my own purposes and I also downloaded some videos with yt-dlp, but now both of those things don't work.

My workaround has been to use a shared VPN IP and just download what I can with yt-dlp. I somehow expect that they're not going to unban my IP though. I got sick of all the scam ads I was being forced to watch, and having my comments not post and it not tell me, iv and yt-dlp are so much cleaner ways to watch.

Thanks anyway for the work that everyone on the yt-dlp project has done so far :> I hope there's some fix at some point <3

@manfromdownunder
Copy link

Your IP has been blacklisted by YouTube. Use a proxy with the following command:
yt-dlp --proxy "userProxy:passwordProxy@ipProxy" URL

VPNs and proxies WILL NOT work as YouTube actively blocks them as well.

I can categorically say that you are wrong about this. When all of this blocking began and video's were unavailable it took 10 mins to setup a second wireguard interface to route invidious through and the problem is no more (for my instance at least). You tube is blocking non-residential IP's much like Netflix and other services do. There are plenty of ways around this currently.

@rjdg14
Copy link

rjdg14 commented Jun 24, 2024

Your IP has been blacklisted by YouTube. Use a proxy with the following command:
yt-dlp --proxy "userProxy:passwordProxy@ipProxy" URL

VPNs and proxies WILL NOT work as YouTube actively blocks them as well.

I can categorically say that you are wrong about this. When all of this blocking began and video's were unavailable it took 10 mins to setup a second wireguard interface to route invidious through and the problem is no more (for my instance at least). You tube is blocking non-residential IP's much like Netflix and other services do. There are plenty of ways around this currently.

I don't think it's possible for sites to block VPNs completely but rather only IP addresses known to be used by VPNs. I have noticed that an increasing number of websites seem to be curbing access from a lot of the popular public VPN servers in the past couple of years or so, yet some of the less popular public servers or private VPN servers still work in many cases.

I think Netflix actively targets known VPNs to try and prevent circumvention of georestrictions. Other sites which seem to block access from a lot of VPNs, for example RateYourMusic and Deviantart, I don't think intentionally block them, but rather blacklist IP addresses that they've detected to be used for automated traffic, which a lot of the popular VPNs have been used for in the past. I think YouTube (or rather its video server) is taking the latter approach although I'm not 100% sure. I'm unsure if YouTube's new blocking measures are designed to prevent crawling from bandwidth consuming AI scrapers or if they are designed to stop the use of yt-dlp, but I think the former is the main reason.

The workaround for a lot of affected Invidious servers seems to have been for them to switch to a rotating IPv6 range, which allows these instances to make requests from one of many addresses, meaning that the IP addresses in use by them avoid automated (excessive) traffic detection by YouTube's servers.

@QGB
Copy link

QGB commented Jun 24, 2024

okK7oSTe2EQ: Sign in to confirm you’re not a bot. This helps protect our community. Learn more File "/home/qgb/anaconda3/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 734, in extr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external issue Issue with an external tool site-bug Issue with a specific website
Projects
None yet
Development

No branches or pull requests