datalad_next.patches.replace_ora_remote

Patch datalad.distributed.ora_remote.ORARemote

This patch replaces the class datalad.distributed.ora_remote.ORARemote with an updated version that should work properly on Linux, OSX, and Windows.

The main difference to the original code is that all path-operations are performed on URL-paths. Those are represented by instances of PurePosixPath. All subclasses of BaseIO, i.e. LocalIO, SSHRemoteIO, and HTTPRemoteIO, are extended to contain the method url2transport_path(). This method converts an URL-path into the correct path for the transport, i.e. the IO abstraction.

Before methods on a subclass of BaseIO that require a path are called, the generic URL-path is converted into the correct path for the IO-class by calling url2transport_path() on the respective IO-class.

The patch keeps changes to the necessary minimum. That means the source is mostly identical to the original. Besides the changes described above, more debug output was added.

NOTE: this patch only provides ORARemote. The patches that add a url2transport_path()-method to LocalIO and to HTTPRemoteIO are contained in module datalad_next.patches.add_method_url2localpath. The reason to keep them separate is that the patch from module datalad_next.patches.replace_create_sibling_ria require them as well. For SSHRemoteIO the method is included in the patch definition of SSHRemoteIO, which is contained in the module datalad_next.patches.replace_sshremoteio.

class datalad_next.patches.replace_ora_remote.ORARemote(annex)[source]

Bases: SpecialRemote

This is the class of RIA remotes.

checkpresent(key)[source]

Requests the remote to check if a key is present in it.

Parameters:

key (str) --

Returns:

True if the key is present in the remote. False if the key is not present.

Return type:

bool

Raises:

RemoteError -- If the presence of the key couldn't be determined, eg. in case of connection error.

close()[source]
dataset_tree_version = '1'
static get_layout_locations(base_path, dsid)[source]
get_store()[source]

checks the remote end for an existing store and dataset

Furthermore reads and stores version and config flags, layout locations, etc. If this doesn't raise, the remote end should be fine to work with.

getcost()[source]

Requests the remote to return a use cost. Higher costs are more expensive.

cheapRemoteCost = 100 nearlyCheapRemoteCost = 110 semiExpensiveRemoteCost = 175 expensiveRemoteCost = 200 veryExpensiveRemoteCost = 1000 (taken from Config/Cost.hs)

Returns:

Indicates the cost of the remote.

Return type:

int

initremote()[source]

Gets called when git annex initremote or git annex enableremote are run. This is where any one-time setup tasks can be done, for example creating the remote folder. Note: This may be run repeatedly over time, as a remote is initialized in different repositories, or as the configuration of a remote is changed. So any one-time setup tasks should be done idempotently.

Raises:

RemoteError -- If the remote could not be initialized.

property io
known_versions_dst = ['1']
known_versions_objt = ['1', '2']
object_tree_version = '2'
prepare()[source]

Tells the remote that it's time to prepare itself to be used. Gets called whenever git annex is about to access any of the below methods, so it shouldn't be too expensive. Otherwise it will slow down operations like git annex whereis or git annex info.

Internet connection can be established here, though it's recommended to defer this until it's actually needed.

Raises:

RemoteError -- If the remote could not be prepared.

property push_io
remove(key)[source]

Requests the remote to remove a key's contents.

Parameters:

key (str) --

Raises:

RemoteError -- If the key couldn't be deleted from the remote.

transfer_retrieve(key, filename)[source]

Get the file identified by key from the remote and store it in local_file.

While the transfer is running, the remote can repeatedly call annex.progress(size) to indicate the number of bytes already stored. This will influence the progress shown to the user.

Parameters:
  • key (str) -- The Key to get from the remote.

  • local_file (str) -- Path where to store the file. Note that in some cases, local_file may contain whitespace.

Raises:

RemoteError -- If the file could not be received from the remote.

transfer_store(key, filename)[source]

Store the file in local_file to a unique location derived from key.

It's important that, while a Key is being stored, checkpresent(key) not indicate it's present until all the data has been transferred. While the transfer is running, the remote can repeatedly call annex.progress(size) to indicate the number of bytes already stored. This will influence the progress shown to the user.

Parameters:
  • key (str) -- The Key to be stored in the remote. In most cases, this is going to be the remote file name. It should be at least be unambiguously derived from it.

  • local_file (str) -- Path to the file to upload. Note that in some cases, local_file may contain whitespace. Note that local_file should not influence the filename used on the remote.

Raises:

RemoteError -- If the file could not be stored to the remote.

verify_ds_in_store()[source]

Check whether the dataset exists in store and reports a layout version we know

The layout is recorded in 'dataset_somewhere_beneath_base_path/ria-layout-version.' If the version found on the remote end isn't supported and force-write isn't configured, sets the remote to read-only operation.

verify_store()[source]

Check whether the store exists and reports a layout version we know

The layout of the store is recorded in base_path/ria-layout-version. If the version found on the remote end isn't supported and force-write isn't configured, sets the remote to read-only operation.

whereis(key)[source]

Asks the remote to provide additional information about ways to access the content of a key stored in it, such as eg, public urls. This will be displayed to the user by eg, git annex whereis. Note that users expect git annex whereis to run fast, without eg, network access.

Parameters:

key (str) --

Returns:

Information about the location of the key, eg. public urls.

Return type:

str

datalad_next.patches.replace_ora_remote.canonify_url(url: str | None)[source]

For file URLs on windows: put the drive letter into the path component

datalad_next.patches.replace_ora_remote.de_canonify_url(url: str | None)[source]

For file URLs on windows: put the drive letter into the netloc component