fs

Functions

archive_directory(dir_, tar_path[, split])

Create tar archive from directory and optionally split it into parts of specified size.

change_directory_at_index(path, dir_name, ...)

Change directory name in path by index.

clean_dir(dir_[, ignore_errors])

Recursively delete a directory tree, but save root directory.

copy_dir_recursively(src_dir, dst_dir[, ...])

rtype

List[str]

copy_file(src, dst)

Copy file from one path to another, if destination directory doesn't exist it will be created.

dir_empty(dir)

Check whether directory is empty or not.

dir_exists(dir)

Check whether directory exists or not.

dirs_filter(input_path, check_function)

Generator that yields paths to directories that meet the requirements of the check_function.

dirs_with_marker(input_path, markers[, ...])

Generator that yields paths to directories that contain markers files.

download(url, save_path[, cache, progress, ...])

Load image from url to host by target path.

ensure_base_path(path)

Recursively create parent directory for target path.

file_exists(path)

Check whether file exists or not.

get_directory_size(dir_path)

Get the size of a directory.

get_file_ext(path)

Extracts file extension from a given path.

get_file_hash(path)

Get hash from target file.

get_file_name(path)

Extracts file name from a given path.

get_file_name_with_ext(path)

Extracts file name with ext from a given path.

get_file_size(path)

Get the size of a file.

get_subdirs(dir_path)

Get list containing the names of the directories in the given directory.

hardlink_or_copy_file(src, dst)

Creates a hard link pointing to src named dst.

hardlink_or_copy_tree(src, dst)

Creates a hard links pointing to src named dst files recursively.

is_on_agent(remote_path)

list_dir_recursively(dir[, include_subdirs, ...])

Recursively walks through directory and returns list with all file paths.

list_files(dir[, valid_extensions, ...])

Returns list with file paths presented in given directory.

list_files_recursively(dir[, ...])

Recursively walks through directory and returns list with all file paths.

log_tree(dir_path, logger[, level])

Get tree for target directory and displays it in the log.

mkdir(dir[, remove_content_if_exists])

Creates a leaf directory and all intermediate ones.

parse_agent_id_and_path(remote_path)

rtype

int

remove_dir(dir_)

Recursively delete a directory tree.

remove_junk_from_dir(dir)

Cleans the given directory from junk files and dirs (e.g.

silent_remove(file_path)

Remove file which may not exist.

string_to_byte_size(string)

Returns integer representation of byte size from string representation.

touch(path)

Sets access and modification times for a file.

tree(dir_path)

Get tree for target directory.

unpack_archive(archive_path, target_dir[, ...])

Unpacks archive to the target directory, removes junk files and directories.

Description

archive_directory(dir_, tar_path, split=None)[source]

Create tar archive from directory and optionally split it into parts of specified size.

Parameters
dir : str

Target directory path.

tar_path : str

Path for output tar archive.

split : Union[int, str]

Split archive into parts of specified size (in bytes) or size with suffix (e.g. ‘1Kb’ = 1024, ‘1Mb’ = 1024 * 1024). Default is None.

Returns

None or list of archive parts if split is not None

Return type

Union[None, List[str]]

Usage example
from supervisely.io.fs import archive_directory
# If split is not needed.
archive_directory('/home/admin/work/projects/examples', '/home/admin/work/examples.tar')

# If split is specified.
archive_parts_paths = archive_directory('/home/admin/work/projects/examples', '/home/admin/work/examples/archive.tar', split=1000000)
print(archive_parts_paths) # ['/home/admin/work/examples/archive.tar.001', '/home/admin/work/examples/archive.tar.002']
change_directory_at_index(path, dir_name, dir_index)[source]

Change directory name in path by index. If you use counting from the end, keep in mind that if the path ends with a file, the file will be assigned to the last index.

Parameters
path : str

The original path

dir_name : str

Directory name

dir_index : int

Index of the directory we want to change, negative values count from the end

Returns

New path

Return type

str

Raises

IndexError – If the catalog index is out of bounds for a given path

Usage example
import supervisely as sly

input_path = 'head/dir_1/file.txt'
new_path = sly.io.fs.change_directory_at_index(input_path, 'dir_2', -2)

print(new_path)
clean_dir(dir_, ignore_errors=True)[source]

Recursively delete a directory tree, but save root directory.

Parameters
dir : str

Target directory path.

Ignore_errors

Ignore possible errors while removes directory content.

Ignore_errors

bool

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import clean_dir
clean_dir('/home/admin/work/projects/examples')
copy_file(src, dst)[source]

Copy file from one path to another, if destination directory doesn’t exist it will be created.

Parameters
src : str

Source file path.

dst : str

Destination file path.

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import copy_file
copy_file('/home/admin/work/projects/example/1.png', '/home/admin/work/tests/2.png')
dir_empty(dir)[source]

Check whether directory is empty or not.

Parameters
dir : str

Target directory path.

Returns

True if directory is empty, False otherwise.

Return type

bool

Usage example
from supervisely.io.fs import dir_empty
dir_empty('/home/admin/work/projects/examples') # False
dir_exists(dir)[source]

Check whether directory exists or not.

Parameters
dir : str

Target directory path.

Returns

True if directory exists, False otherwise.

Return type

bool

Usage example
from supervisely.io.fs import dir_exists
dir_exists('/home/admin/work/projects/examples') # True
dir_exists('/home/admin/work/not_exist_dir') # False
dirs_filter(input_path, check_function)[source]

Generator that yields paths to directories that meet the requirements of the check_function.

Parameters
input_path : str

path to the directory in which the search will be performed

check_function : Callable

function to check that directory meets the requirements and returns bool

Usage example
import supervisely as sly

input_path = '/home/admin/work/projects/examples'

# Prepare the check function.

def check_function(directory) -> bool:
    images_dir = os.path.join(directory, "images")
    annotations_dir = os.path.join(directory, "annotations")

    return os.path.isdir(images_dir) and os.path.isdir(annotations_dir)

for directory in sly.fs.dirs(input_path, check_function):
    # Now you can be sure that the directory meets the requirements.
    # Do something with it.
    print(directory)
Return type

Generator[str, None, None]

dirs_with_marker(input_path, markers, check_function=None, ignore_case=False)[source]

Generator that yields paths to directories that contain markers files. If the check_function is specified, then the markered directory will be yielded only if the check_function returns True. The check_function must take a single argument - the path to the markered directory and return True or False.

Parameters
input_path : str

path to the directory in which the search will be performed

markers : Union[str, List[str]]

single marker or list of markers (e.g. ‘config.json’ or [‘config.json’, ‘config.yaml’])

check_function : Callable

function to check that directory meets the requirements and returns bool

ignore_case : bool

ignore case when searching for markers

Usage example
import supervisely as sly

input_path = '/home/admin/work/projects/examples'

# You can pass a string if you have only one marker.
# markers = 'config.json'

# Or a list of strings if you have several markers.
# There's no need to pass one marker in different cases, you can use ignore_case=True for this.
markers = ['config.json', 'config.yaml']


# Check function is optional, if you don't need the directories to meet any requirements,
# you can omit it.

def check_function(dir_path):
    test_file_path = os.path.join(dir_path, 'test.txt')
    return os.path.exists(test_file_path)

for directory in sly.fs.dirs_with_marker(input_path, markers, check_function, ignore_case=True):
    # Now you can be sure that the directory contains the markers and meets the requirements.
    # Do something with it.
    print(directory)
Return type

Generator[str, None, None]

download(url, save_path, cache=None, progress=None, headers=None, timeout=None)[source]

Load image from url to host by target path.

Parameters
url : str

Target file path.

url

The path where the file is saved.

cache : FileCache, optional

An instance of FileCache class that provides caching functionality for the downloaded content. If None, caching is disabled.

progress : Progress, optional

Function for tracking download progress.

headers : Dict, optional.

A dictionary of HTTP headers to include in the request.

timeout : int, optional.

The maximum number of seconds to wait for a response from the server. If the server does not respond within the timeout period, a TimeoutError is raised.

Returns

Full path to downloaded image

Return type

str

Usage example
from supervisely.io.fs import download
img_link = 'https://m.media-amazon.com/images/M/MV5BMTYwOTEwNjAzMl5BMl5BanBnXkFtZTcwODc5MTUwMw@@._V1_.jpg'
im_path = download(img_link, '/home/admin/work/projects/examples/avatar.jpeg')
print(im_path)
# Output:
# /home/admin/work/projects/examples/avatar.jpeg

# if you need to specify some headers
headers = {
    'User-Agent': 'Mozilla/5.0',
}
im_path = download(img_link, '/home/admin/work/projects/examples/avatar.jpeg', headers=headers)
print(im_path)
# Output:
# /home/admin/work/projects/examples/avatar.jpeg
ensure_base_path(path)[source]

Recursively create parent directory for target path.

Parameters
path : str

Target dir path.

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import ensure_base_path
ensure_base_path('/home/admin/work/projects/example')
file_exists(path)[source]

Check whether file exists or not.

Parameters
dir : str

Target file path.

Returns

True if file exists, False otherwise.

Return type

bool

Usage example
from supervisely.io.fs import file_exists
file_exists('/home/admin/work/projects/examples/1.jpeg') # True
file_exists('/home/admin/work/projects/examples/not_exist_file.jpeg') # False
get_directory_size(dir_path)[source]

Get the size of a directory.

Parameters
path : str

Target directory path.

Returns

Directory size in bytes

Return type

int

Usage example
from supervisely.io.fs import get_directory_size
dir_size = get_directory_size('/home/admin/work/projects/examples') # 8574563
get_file_ext(path)[source]

Extracts file extension from a given path.

Parameters
path : str

Path to file.

Returns

File extension without name

Return type

str

Usage example
import supervisely as sly

file_ext = sly.fs.get_file_ext("/home/admin/work/projects/lemons_annotated/ds1/img/IMG_0748.jpeg")

print(file_ext)
# Output: .jpeg
get_file_hash(path)[source]

Get hash from target file.

Parameters
path : str

Target file path.

Returns

File hash

Return type

str

Usage example
from supervisely.io.fs import get_file_hash
hash = get_file_hash('/home/admin/work/projects/examples/1.jpeg') # rKLYA/p/P64dzidaQ/G7itxIz3ZCVnyUhEE9fSMGxU4=
get_file_name(path)[source]

Extracts file name from a given path.

Parameters
path : str

Path to file.

Returns

File name without extension

Return type

str

Usage example
import supervisely as sly

file_name = sly.fs.get_file_name("/home/admin/work/projects/lemons_annotated/ds1/img/IMG_0748.jpeg")

print(file_name)
# Output: IMG_0748
get_file_name_with_ext(path)[source]

Extracts file name with ext from a given path.

Parameters
path : str

Path to file.

Returns

File name with extension

Return type

str

Usage example
import supervisely as sly

file_name_ext = sly.fs.get_file_name_with_ext("/home/admin/work/projects/lemons_annotated/ds1/img/IMG_0748.jpeg")

print(file_name_ext)
# Output: IMG_0748.jpeg
get_file_size(path)[source]

Get the size of a file.

Parameters
path : str

File path.

Returns

File size in bytes

Return type

int

Usage example
from supervisely.io.fs import get_file_size
file_size = get_file_size('/home/admin/work/projects/examples/1.jpeg') # 161665
get_subdirs(dir_path)[source]

Get list containing the names of the directories in the given directory.

Parameters
dir_path : str

Target directory path.

Returns

List containing directories names.

Return type

list

Usage example
from supervisely.io.fs import get_subdirs
subdirs = get_subdirs('/home/admin/work/projects/examples')
print(subdirs)
# Output: ['tests', 'users', 'ds1']

Creates a hard link pointing to src named dst. If the link cannot be created, the file will be copied.

Parameters
src : str

Source file path.

dst : str

Destination file path.

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import hardlink_or_copy_file
hardlink_or_copy_file('/home/admin/work/projects/example/1.png', '/home/admin/work/tests/link.txt')

Creates a hard links pointing to src named dst files recursively. If the link cannot be created, the file will be copied.

Parameters
src : str

Source dir path.

dst : str

Destination dir path.

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import hardlink_or_copy_tree
hardlink_or_copy_tree('/home/admin/work/projects/examples', '/home/admin/work/tests/links')
list_dir_recursively(dir, include_subdirs=False, use_global_paths=False)[source]

Recursively walks through directory and returns list with all file paths.

Parameters
path : str

Path to directory.

include_subdirs : bool

If True, subdirectories will be included in the result list.

use_global_paths : bool

If True, absolute paths will be returned instead of relative ones.

Returns

List containing file paths.

Return type

List[str]

Usage example
import supervisely as sly

list_dir = sly.fs.list_dir_recursively("/home/admin/work/projects/lemons_annotated/")

print(list_dir)
# Output: ['meta.json', 'ds1/ann/IMG_0748.jpeg.json', 'ds1/ann/IMG_4451.jpeg.json', 'ds1/img/IMG_0748.jpeg', 'ds1/img/IMG_4451.jpeg']
list_files(dir, valid_extensions=None, filter_fn=None, ignore_valid_extensions_case=False)[source]

Returns list with file paths presented in given directory.

Parameters
dir : str

Target dir path.

dir

str

valid_extensions : List[str]

List with valid file extensions.

filter_fn

Function with a single argument that determines whether to keep a given file path.

ignore_valid_extensions_case : bool

If True, valid extensions will be compared with file extensions in case-insensitive manner.

Returns

List with file paths

Return type

List[str]

Usage example
import supervisely as sly

list_files = sly.fs.list_files("/home/admin/work/projects/lemons_annotated/ds1/img/")

print(list_files)
# Output: ['/home/admin/work/projects/lemons_annotated/ds1/img/IMG_0748.jpeg', '/home/admin/work/projects/lemons_annotated/ds1/img/IMG_4451.jpeg']
list_files_recursively(dir, valid_extensions=None, filter_fn=None)[source]

Recursively walks through directory and returns list with all file paths.

param dir

Target dir path.

param dir

str

param valid_extensions

List with valid file extensions.

type valid_extensions

List[str]

param filter_fn

Function with a single argument that determines whether to keep a given file path.

type filter_fn

returns

List with file paths

rtype

List[str]

Usage example
import supervisely as sly

list_files = sly.fs.list_files_recursively("/home/admin/work/projects/lemons_annotated/ds1/img/")

print(list_files)
# Output: ['/home/admin/work/projects/lemons_annotated/ds1/img/IMG_0748.jpeg', '/home/admin/work/projects/lemons_annotated/ds1/img/IMG_4451.jpeg']
Return type

List[str]

log_tree(dir_path, logger, level='info')[source]

Get tree for target directory and displays it in the log.

Parameters
dir_path : str

Target directory path.

logger : logger

Logger to display data.

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import log_tree
logger = sly.logger
log_tree('/home/admin/work/projects/examples', logger)
mkdir(dir, remove_content_if_exists=False)[source]

Creates a leaf directory and all intermediate ones.

Parameters
dir : str

Target dir path.

dir

str

Remove_content_if_exists

Remove directory content if it exist.

Remove_content_if_exists

bool

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import mkdir
mkdir('/home/admin/work/projects/example')
remove_dir(dir_)[source]

Recursively delete a directory tree.

Parameters
dir : str

Target directory path.

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import remove_dir
remove_dir('/home/admin/work/projects/examples')
remove_junk_from_dir(dir)[source]

Cleans the given directory from junk files and dirs (e.g. .DS_Store, __MACOSX, Thumbs.db, etc.).

Parameters
dir : str

Path to directory.

Returns

List of global paths to removed files and dirs.

Return type

List[str]

Usage example
import supervisely as sly

input_dir = "/home/admin/work/projects/lemons_annotated/"
sly.fs.remove_junk_from_dir(input_dir)
silent_remove(file_path)[source]

Remove file which may not exist.

Parameters
file_path : str

File path.

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import silent_remove
silent_remove('/home/admin/work/projects/examples/1.jpeg')
string_to_byte_size(string)[source]
Returns integer representation of byte size from string representation.

If input is integer, returns the same integer for convenience.

param string

string representation of byte size (e.g. 1.5Kb, 2Mb, 3.7Gb, 4.2Tb) or integer

type string

Union[str, int]

return

integer representation of byte size (or the same integer if input is integer)

rtype

int

raises ValueError

if input string is invalid

Usage example

Return type

int

touch(path)[source]

Sets access and modification times for a file.

Parameters
path : str

Target file path.

Returns

None

Return type

NoneType

Usage example
from supervisely.io.fs import touch
touch('/home/admin/work/projects/examples/1.jpeg')
tree(dir_path)[source]

Get tree for target directory.

Parameters
dir_path : str

Target directory path.

Returns

Tree with directory files and subdirectories

Return type

str

Usage example
from supervisely.io.fs import tree
dir_tree = tree('/home/admin/work/projects/examples')
print(dir_tree)
# Output: /home/admin/work/projects/examples
# ├── [4.0K]  1
# │   ├── [165K]  crop.jpeg
# │   ├── [169K]  fliplr.jpeg
# │   ├── [169K]  flipud.jpeg
# │   ├── [166K]  relative_crop.jpeg
# │   ├── [167K]  resize.jpeg
# │   ├── [169K]  rotate.jpeg
# │   ├── [171K]  scale.jpeg
# │   └── [168K]  translate.jpeg
# ├── [ 15K]  123.jpeg
# ├── [158K]  1.jpeg
# ├── [188K]  1.txt
# ├── [1.3M]  1.zip
# ├── [4.0K]  2
# ├── [ 92K]  acura.png
# ├── [1.2M]  acura_PNG122.png
# ├── [198K]  aston_martin_PNG55.png
# ├── [4.0K]  ds1
# │   ├── [4.0K]  ann
# │   │   ├── [4.3K]  IMG_0748.jpeg.json
# │   │   ├── [ 151]  IMG_0777.jpeg.json
# │   │   ├── [ 151]  IMG_0888.jpeg.json
# │   │   ├── [3.7K]  IMG_1836.jpeg.json
# │   │   ├── [8.1K]  IMG_2084.jpeg.json
# │   │   ├── [5.5K]  IMG_3861.jpeg.json
# │   │   ├── [6.0K]  IMG_4451.jpeg.json
# │   │   └── [5.0K]  IMG_8144.jpeg.json
# │   └── [4.0K]  img
# │       ├── [152K]  IMG_0748.jpeg
# │       ├── [210K]  IMG_0777.jpeg
# │       ├── [210K]  IMG_0888.jpeg
# │       ├── [137K]  IMG_1836.jpeg
# │       ├── [139K]  IMG_2084.jpeg
# │       ├── [145K]  IMG_3861.jpeg
# │       ├── [133K]  IMG_4451.jpeg
# │       └── [136K]  IMG_8144.jpeg
# ├── [152K]  example.jpeg
# ├── [2.4K]  example.json
# ├── [153K]  flip.jpeg
# ├── [ 65K]  hash1.jpeg
# ├── [ 336]  meta.json
# └── [5.4K]  q.jpeg
# 5 directories, 37 files
unpack_archive(archive_path, target_dir, remove_junk=True)[source]

Unpacks archive to the target directory, removes junk files and directories.

Parameters
archive_path : str

Path to the archive.

target_dir : str

Path to the target directory.

remove_junk : bool

Remove junk files and directories. Default is True.

Returns

None

Return type

NoneType

Usage example
import supervisely as sly

archive_path = '/home/admin/work/examples.tar'
target_dir = '/home/admin/work/projects'

sly.fs.unpack_archive(archive_path, target_dir)