Introduction
Google Colab is a powerful tool for data scientists and researchers to work on projects. However, one of the challenges when working on projects that require external files is uploading those files directly from your desktop into your Colab environment. This process can be slow and frustrating. But there are two faster options to upload and access files in Colab: using wget to retrieve files from an external location and mounting your Google Drive to access files there. In this guide, we will discuss how to quickly upload files into Google Colab using these two options.
When Uploading Files is Slow in Google Colab
When you are working on projects that require external files in Google Colab, you may notice that uploading files directly from your desktop into your Colab environment is slow. This can be frustrating, especially when you are working on a project that requires multiple files. However, there are two faster options to upload and access files in Colab: using wget to retrieve files from an external location and mounting your Google Drive to access files there.
Using either wget to download a file from an external location or Google Drive comes with a significant benefit: because the file is persistent, you can access it in multiple sessions without having to re-upload the file into Colab.
In this guide, we will discuss how to quickly upload files into Google Colab with wget and Google Drive.
Without further ado, let’s get started!
Option #1: Mount Google Drive
You can upload files to Google Drive then download them into Google Colab. This allows you to permanently store your files in the cloud. Google Drive has an official integration with Colab. With a few lines of code, you can mount and retrieve files from Google Drive in Colab.
While you still need to upload your files on your internet connection, you only need to do this once. This is faster than manually uploading files into Colab, where you need to upload files every time you start a new session.
First, upload the files you want to use in Colab notebooks to your Google Drive. We recommend uploading the files in a folder so that you can mount the specific folder.
Here is an example of a folder called bottles in a Google Drive. In this scenario, I’m loading an image dataset into Colab to train a computer vision model. Computer vision datasets can be many thousands of images which makes this integration especially useful.
To mount your Google Drive into Colab, create a new Colab cell and add the following code:
from google.colab import drive
from google.colab import files
drive.mount("/content/drive")You can only mount your full Drive; you cannot mount specific folders.
When you run this code, you will be asked to confirm you want to connect to Google Drive:
You will then be asked to grant the requisite permissions:
💡
Note: You should only download files from Google Drive if you trust the code in the notebook, since granting permission gives read and write access to the files in Drive.
Then, run:
files.upload()This will download the files from the folder into your Google Colab environment, allowing you to use them in your notebook.
For example, we can open a file from our Drive using:
from IPython.display import Image
Image('/content/drive/bottles/images/example.png')
Here is the result:
We successfully downloaded our files into Colab.
Option #2: Upload with wget
If you have a file available on an external website – a CDN, for example – you can download the files into Google Colab with wget. This is faster than uploading files directly into Google Colab because Colab’s back-end can download the files directly into the Colab back-end.
You can download a file with wget using the following code:
!wget
Where example.com is the URL of the file that you are downloading.
If you have downloaded a zip file, you can unzip it with:
!unzip file.zipConclusion
In this guide, we walked through two ways to quickly upload files into Google Colab:
- Mount a Google Drive folder and download files from there, and;
- Download files from wget that are available from an external source (i.e. a CDN).
Both of these options offer a significant speedup over manually uploading files directly into Colab. We hope this guide has been helpful in getting you started with using Google Colab.
Frequently Asked Questions
Q1: Can I mount a specific folder on my Google Drive in Colab?
No, you cannot mount a specific folder on your Google Drive in Colab. You can only mount your full Drive.
Q2: Can I upload files from an external source other than Google Drive or a CDN?
No, these two options are the fastest ways to upload files into Colab. However, you can also manually upload files from your desktop or another location using the “Upload” button in the Colab interface.
Q3: How do I grant permissions to Google Drive in Colab?
When you run the code to mount your Google Drive, you will be asked to confirm you want to connect to Google Drive. You will then be asked to grant the requisite permissions. This will give read and write access to the files in Drive.
Q4: Can I download multiple files at once from Google Drive in Colab?
Yes, you can download multiple files at once from Google Drive in Colab by selecting the files you want to download and using the “Download” button.
Q5: Is there a limit to the size of files I can upload into Colab?
No, there is no limit to the size of files you can upload into Colab. However, the time it takes to upload and download large files may be longer.
