How URL shorteners work
The purpose of a URL shortener is to replace a log URL (e.g: http://www.zscaler.com/downloadwhitepaper_stateofweb-q4-2009.html) with a shorter one (e.g: http://bit.ly/cikl0z). When a user clicks on http://bit.ly/cikl0z, he is redirected to http://www.zscaler.com/downloadwhitepaper_stateofweb-q4-2009.html via an HTTP 301 redirection:
GET /cikl0z HTTP/1.1
Host: bit.ly
HTTP/1.1 301 Moved
Location: http://www.zscaler.com/downloadwhitepaper_stateofweb-q4-2009.html
----------------------------------------------------------
GET /downloadwhitepaper_stateofweb-q4-2009.html HTTP/1.1
Host: www.zscaler.com
HTTP/1.1 200 OK
The browser made two requests: one to http://bit.ly/cikl0 and one to http://www.zscaler.com/downloadwhitepaper_stateofweb-q4-2009.htmlHost: bit.ly
HTTP/1.1 301 Moved
Location: http://www.zscaler.com/downloadwhitepaper_stateofweb-q4-2009.html
----------------------------------------------------------
GET /downloadwhitepaper_stateofweb-q4-2009.html HTTP/1.1
Host: www.zscaler.com
HTTP/1.1 200 OK
Existing defense mechanisms
All the existing in-browser (Google Safe Browsing in Firefox, Opera's Fraud Protection, etc.) or external (IDS, proxy, etc.) URL scanners are applied on both the initial short and redirected long URL requests. If the long URL is a known malicious site, it will be stopped whether or not the the user clicks directly on the long URL, or on a shortened URL.
Firefox Safe Google Browsing warning on a URL after a redirection
Also, content inspection (Antivirus, Deep Packet inspection, etc.) is applied on both requests.
The use of URL shorteners and redirections does not require any new security inspection. All of the web browser security tools in place prior to the use of URL shorteners are still relevant.
Hiding the real URL
The main argument against URL shortening services is that users don't know which domain they are being redirected to. In our previous example, users see the bit.ly host name in the link address, and do not know that they will be redirected to www.zscaler.com until after they click on the link. After the redirection, the ultimate destination URL can be seen in the web browser address bar.
The long URL is displayed in the browser address bar after redirection
How many people know the difference between a good URL and a bad URL? Even then, how can anyone be sure that a site won't serve malicious content. Many perfectly legitimate websites (Redcross, Indian Governmental websites, etc.) have been hacked and can contain an infamous hidden iframe to spread malware. Well-known websites are no longer necessarily safer than unknown or new sites. Simply using the reputation of the hostname for deciding whether a URL is safe or not is not a good idea.
In a post Michael wrote a year ago, he checked 100,000 TinyURL (URL shortener service) urls. He did not find any link to a malicious executable, no phishing sites, and really few redirections to malicious content.
I believe the danger of URL shorteners has been overblown, mainly based on the idea that individuals are in a position to determine if a website is dangerous or not simply by looking at the final URL. Users are far better off relying on antivirus, URL blacklists and regular browser updates for security. And these tools work just fine or shortened URLs as well.
- Julien
Posts
4 comments:
John, you wrote a great article with much needed insight. I am trying to understand how to shorten a URL when posting to Twitter. Best, Rose Gabriele
DNS poisoning has long been a security threat - the idea that you can alter a benign name to redirect to a malicious IP. URL shortners put an additional layer that can be compromised on top of that. A nice SQL Injection flaw in a shortner service and a whole wealth of formerly benign URLs now point to malicious addresses. Do you think that every URL shortner has perfect security around the poisoning of their URLs?
It perhaps isn't the greatest risk in the world, but it does add an additional layer of complexity that can go wrong, and qualitatively that does mean the security threat is somewhat elivated (though I don't lose sleep over that).
Per the in browser save browsing, those aren't bullet proof, and while the NSS report has gotten a fair amount of criticism (much of which they addressed in their methodology this year, but any report kind to MS is going to seem shady to many despite how well thought out the methodology), there is no reason to think that the "safe browsing" features of each browser are equally effective (Opera specifically is of question). Also, let's not kid ourselves, if safe browsing was adequate protection there would not be web delivered malware. It is a decent defense in depth, but not adequate by itself.
Personally though, I won't click on any URL without knowing where it will go, not for software security reasons, but rather mental security reasons. Many people on mailing lists I frequented in college delighted in directing folks to content to rattle the mental stability of their peers, and for that reason I have a strong distrust of any obfuscated destination despite more than a decade having passed since that time. Some behaviors just become engrained.
Joshbw: I think the idea that we can measure the potential security threat of a URL by looking at the domain name does not hold anymore. More and more well-known website are infected with a malicious iframe, for example. I think no domain can be fully trusted anymore. It is better to rely on content and inspection and blacklist. Even if these tools are far from perfect, they offer a better protection that just looking at the URL.
I believe most users thinks search results displayed by Google are safe. This is not the case. Users have to suspect all links by default. Relying solely on how the URL look like is too dangerous.
Julien - I think you missed the majority of my point by focusing on my rather glib closing comment.
I don't think URL shortners represent a specific security threat because of the inability to inspect the URL. Rather I think they present an additional area to compromise a previously good URL by redirecting to a malicious site to a malicious site. You are adding an additional layer of name resolution, a nice big centralized one, to the mix (and many of these services are not exactly developed with SDL-like security scrutiny). That is my primary security beef with it.
To demonstrate, say you set up a shortned URL "http://bit.ly/cikl0z" that redirects to "http://www.zscaler.com/downloadwhitepaper_stateofweb-q4-2009.html ". If bit.ly has any SQL injection flaws into their database of URL pairs one nicely crafted SQL update and your previously safe shortened URL, and everyone elses, now points to a malicious site. You are creating a new layer of name resolution to attack, one that is only necessary because twitter won't count just the text of an anchor tag rather than the reference in its 140 character limit. Supporting the limitations of a micro-blogging site is *not* a good reason to add an additional layer of complexity to name resolution.
Now per my glib comment at the end of my last response - there are a lot of completely legitimate scenarios where knowing the destination of a URL makes sense - is the URL ok to look at at work during a break or is it better to wait until I am home (amazon - probably, 4chan - not so much), does it go to yahoo.com or yahhoo.com, etc. There are a lot more reasons to want to know where a URL points to than whether it might be hosting malware. Smart browsing filters are not a replacement for human judgement, but a complement to them, and it is far better to have both than just one. I have no illusions that most people look at the URL and make such judgement calls, but I don't think it is justified handicapping those that do for the sake of twitter.
Post a Comment