A Startup, by the very nature of what it is, often involves a bit of risk taking, a mindset that can evaluate the potential risks against the possible rewards and benefits. Sometimes temporary shortcuts may be required to reach the ultimate goal. This is why in the case of startups and under certain guidelines, I believe data Scraping should be considered.

What is data scraping?

According to one definition, Web scraping is a term for “various methods used to collect information from across the Internet. Generally, this is done with software that simulates human Web surfing to collect specified bits of information from different websites.”

That same source notes that the practice has drawn controversy, and in some cases, legal challenges, because “the terms of use for some websites do not allow certain kinds of data mining.”

Why shortcuts are okay

Even large companies take shortcuts. While they may have internal rules governing how products are developed and how money is spent, many companies also form incubators which are exempt from these internal rules. Incubators are established to work outside the scope of some of these existing rules and regulations so they can truly think outside the box and innovate without being constrained by processes and procedures.

Since a startup needs to think outside the box, and is hindered, perhaps not by corporate rules and regulations but by financial and time constraints, I believe that in some cases, data scraping is an acceptable shortcut a startup may want to consider.

The conditions

In order for data scraping to be acceptable, in my opinion, it cannot relate to any individual’s personal information. It should also be considered only as a temporary measure with an understanding that your own data will be developed and acquired as an asset along the way.

Understanding that a startup often needs a business model, or a proof of concept in order to gain investors, I would suggest these as models under which data scraping may make sense.

To scrape or not

If you are considering data scraping, I recommend you speak with a lawyer to understand the potential risks and consequences. A good lawyer will likely tell you not to do it, but they will also outline their reasoning which will allow you to determine whether the risk supports the potential reward.

I also recommend that if you decide to go ahead to look at using a server in a country whose laws are not as rigidly against scraping so that your personal risk is less.

Consider the data temporary

Data obtained from scraping should be a temporary measure, as noted above. It is also important to understand that any data you have access to at one moment, you may lose the next as systems are changed and upgraded.

Depending on the source of the data and its value, it may be worth negotiating an agreement with the company, a partnership of sorts, that will allow you access to the data legitimately for short or long-term use. Of course, there will need to be a reason for them to allow you this access, but make them understand data in terms of it being an asset and that you are adding value to an asset that is already there.

Don’t forget open source

Depending on the data that you require, remember that open source offers data that is legally and openly available and may provide what you need to get your startup off the ground, so it is definitely worth checking out first.

If you want to learn more about the startup mindset and how you can use it to grow your own startup, check out the free Startup Technical Plan course on myctofriend.co.