sourceplus_sdk package

Submodules

sourceplus_sdk.libs module

class sourceplus_sdk.libs.AsyncCounter[source]

Bases: object

async decrement()[source]
async get()[source]
async increment()[source]

sourceplus_sdk.main module

async sourceplus_sdk.main.download_image(client: AsyncClient, url: str, semaphore: Semaphore, success_counter: AsyncCounter, failure_counter: AsyncCounter, destination_folder: str, api_key: str)[source]

Downloads an image from the given URL, using the semaphore to control the number of parallel downloads.

Parameters:

  • client (httpx.AsyncClient) – The HTTP client to use for downloading the image.

  • url (str) – The URL of the image to download.

  • semaphore (asyncio.Semaphore) – The semaphore to control the number of parallel downloads.

  • success_counter (AsyncCounter) – The counter to keep track of the number of successful downloads.

  • failure_counter (AsyncCounter) – The counter to keep track of the number of failed downloads.

  • destination_folder (str) – The path to the folder where the image will be saved.

  • api_key (str) – The Source+ download key.

sourceplus_sdk.main.download_images(file_path: str, output_folder: str, num_download_jobs: int = 50, limit_images: int = -1, url_column_name: str = 'url', show_progress: bool = False)[source]

Download images from the given file path to the destination folder.

Parameters:

  • file_path (str) – The path to the file containing the URLs of the images to download.

  • output_folder (str) – The path to the folder where the images will be saved.

  • num_download_jobs (int, optional) – The maximum number of parallel downloads. Defaults to 50.

  • limit_images (int, optional) – The maximum number of images to download. Defaults to -1, which means all images.

  • url_column_name (str, optional) – The name of the column containing the URLs in the file. Defaults to “url”.

  • show_progress (bool, optional) – Whether to show a progress bar. Defaults to False.

sourceplus_sdk.main.main(args)[source]

Wrapper allowing download_images() to be called with string arguments in a CLI fashion

Parameters:

args (List[str]) – command line parameters as list of strings

sourceplus_sdk.main.parse_args(args)[source]

Parse command line parameters

Parameters:

args (List[str]) – command line parameters as list of strings (for example ["--help"]).

Returns:

command line parameters namespace

Return type:

argparse.Namespace

async sourceplus_sdk.main.progress_monitor(success_counter: AsyncCounter, failure_counter: AsyncCounter, total_images: int)[source]

Monitors the progress of the downloads.

Parameters:

  • success_counter (AsyncCounter) – The counter to monitor.

  • failure_counter (AsyncCounter) – The counter to monitor.

  • total_images (int) – The total number of images to download.

sourceplus_sdk.main.prompt_destination_folder_creation(destination_folder: str)[source]

Prompts the user to create the destination folder if it does not exist.

Parameters:

destination_folder (str) – The path to the destination folder.

sourceplus_sdk.main.prompt_for_download_key()[source]

Prompts the user for the download key.

Returns:

The download key.

Return type:

str

sourceplus_sdk.main.run()[source]

Calls main() passing the CLI arguments extracted from sys.argv

This function can be used as entry point to create console scripts with setuptools.

sourceplus_sdk.main.setup_logging(loglevel)[source]

Setup basic logging

Parameters:

loglevel (int) – minimum loglevel for emitting messages

async sourceplus_sdk.main.start_image_downloads(df: DataFrame, max_parallel_downloads: int, max_images: int, destination_folder: str, api_key: str, show_progress: bool = False)[source]

Starts downloading images from the given DataFrame.

Parameters:

  • df (pd.DataFrame) – The DataFrame containing the URLs of the images to download.

  • max_parallel_downloads (int) – The maximum number of parallel downloads.

  • max_images (int) – The maximum number of images to download.

  • destination_folder (str) – The path to the folder where the images will be saved.

  • api_key (str) – The Source+ download key.

  • show_progress (bool, optional) – Whether to show a progress bar. Defaults to False.

sourceplus_sdk.main.validate_args(file_path: str, output_folder: str, num_download_jobs: int)[source]

Validates the input arguments.

Parameters:

  • file_path (str) – The path to the file containing the URLs of the images to download.

  • output_folder (str) – The path to the folder where the images will be saved.

  • num_download_jobs (int) – The maximum number of parallel downloads.

sourceplus_sdk.main.validate_file(file_path: str, url_column_name: str, limit_images: int = -1)[source]

Validates the file path.

Parameters:

  • file_path (str) – The path to the file containing the URLs of the images to download.

  • url_column_name (str) – The name of the column containing the URLs in the file.

Returns:

The DataFrame containing the URLs of the images to download.

Return type:

pd.DataFrame

Module contents