Is Web Scraping Legal?

The short answer is that collecting publicly available data is generally lawful in many jurisdictions, while the specific things that get people sued are contract and personal data, not the act of collection itself.

This is a summary of the publicly documented position in two major jurisdictions, with sources. It is not legal advice, and the answer depends on what you collect, where you are, and what you agreed to.

United States: the computer fraud question

The statute that dominated early scraping litigation was the Computer Fraud and Abuse Act, which made it a violation to access a computer without authorisation or in a way that exceeds authorised access.

Two decisions narrowed that considerably.

Van Buren v. United States, 593 U.S. 374 (2021). The Supreme Court held, 6-3, that the CFAA's "exceeds authorized access" language covers a gates-up-or-down question. A person who is entitled to access a system does not violate the statute by accessing it for an improper purpose. The decision rejected the broader reading that would have made misuse of information a federal crime.

hiQ Labs, Inc. v. LinkedIn Corp., 938 F.3d 985 (9th Cir. 2019). The Ninth Circuit held that scraping publicly available data does not constitute access "without authorization" under the CFAA, because a public website does not gate access in the way the statute contemplates. The decision was vacated and remanded by the Supreme Court in light of Van Buren, and the Ninth Circuit affirmed its earlier reasoning on remand on April 18, 2022.

The practical reading of those two together: the CFAA is a weak instrument against scraping public data, and the litigation risk sits elsewhere.

Where hiQ still lost. It is worth being explicit, because the case is often cited as a scraping victory. hiQ lost on remand at the district court level on the remaining claims, including breach of contract and trespass to chattels, and the parties settled. The precedent that survived is about the CFAA. The commercial outcome went the other way.

The real risk in the US: contract

The claims that actually succeed against scrapers are usually:

Breach of contract. Terms of service are a contract, and using a site is generally treated as acceptance. Violating a prohibition in the terms is a contractual breach, enforceable even where the CFAA does not apply, as hiQ's own outcome shows.

Trespass to chattels. Where collection imposes a measurable burden on the target's systems.

Copyright. Copying substantial creative content, as distinct from facts.

Trademark and misappropriation. Reusing branded content in a way that misleads.

The pattern: the data being public does not neutralise a contract you accepted, and it does not make copying protected content permissible.

European Union

The framework is different and turns on data protection rather than access.

GDPR. Applies whenever personal data is processed, regardless of whether it was public. Scraping publicly visible personal data is processing, so it requires a lawful basis, and the obligations follow: notice or a documented basis, data minimisation, retention limits, security, and the rights of data subjects including erasure.

Public accessibility does not remove any of those. A published email address is still personal data.

Database rights. Where they exist, extracting a substantial part of a database can infringe a sui generis database right, which is a separate claim from copyright.

Terms and contract. The same contractual exposure as in the US.

National law. Member states differ on specifics, and some have additional restrictions on collection.

The practical consequence for an EU-facing project: identify the personal data in your dataset before you build, not after. The framework in Data Collection Ethics for Engineering Teams starts there for a reason.

What is not in dispute

A few things are clear enough to state without qualification.

• Behind a login you are not entitled to use is the strongest case against you, in every jurisdiction.
• Ignoring an explicit technological access control, such as circumventing authentication, moves a matter from contract to the computer misuse statutes.
• Collecting special category personal data has the highest exposure and rarely has a defensible basis for scraping.
• Degrading a service for its users creates a harm that exists regardless of the access question.
• Terms of service are enforceable, and the fact that a prohibition is inconvenient does not make it inapplicable.

A working summary

For a project that collects public, non-personal data at a rate the source tolerates, consistent with the source's terms, the practical risk is low in both jurisdictions.

Risk rises with each of the following, roughly in this order: collecting personal data without a basis, ignoring a prohibition in the terms, circumventing an access control, collecting at a rate that burdens the source, and republishing substantial creative content.

The questions to answer before building:

1. Is any of the data personal? Under which regime, and on what basis?
2. What do the source's terms say, and does the collection comply?
3. Is there an access control, and is it being circumvented?
4. What rate, and does it affect the source's other users?
5. What is retained, for how long, and who is accountable?

Sources

• Van Buren v. United States, 593 U.S. 374 (2021)
• hiQ Labs, Inc. v. LinkedIn Corp., 938 F.3d 985 (9th Cir. 2019), affirmed on remand April 18, 2022
• Regulation (EU) 2016/679 (GDPR), in particular Articles 5, 6 and 17
• Directive 96/9/EC on the legal protection of databases

Consult a lawyer for your specific situation. This page describes the public record, not your position under it.

For the ethical framework that sits alongside the legal one, see Data Collection Ethics for Engineering Teams.