Microsoft Azure Data Lake Gen 1 Storage Connector

This adapter enables communication with first generation Microsoft Azure Date Lakes (based on Apache Hadoop).

Properties

Operation

Defines the operation executed by the adapter

Possible values:

AppendFile: Append a file to an already existing file.
ConcatenateFiles: Concatenate a specific list of files into a destination file.
The source files will be deleted, if the concatenation succeeds. The target file will be created, if it does not exist. Otherwise, the content will be concatenated.
CreateFolder: Create a folder.
DeleteFile: Delete a file.
DeleteFolder: Recursively delete a folder and its complete content.
EnumerateDirectory: List all files and folders from a specified folder path.
GetMetadata: Get metadata of a file or folder.
ReadFile: Read a file.
RenameFile: Rename a file. If the target file exists (a file with a same name), the source file will be only renamed, if the parameter overwrite is enabled.
RenameFolder: Rename a folder. If the target folder exists, the source folder will be moved into the target folder.
SetPermission: Set the permissions of the specified file or directory.
UploadFile: Create a new file.

Parameters

`Adapter`	Main adapter class (do not change!) Possible values: de.softproject.integration.adapter.microsoft.azure.datalake.MicrosoftAzureDataLakeStoreConnector: main class (default)
`authEndpoint`	Required: OAuth 2.0 token endpoint (v1) Possible values: String containing a valid endpoint, e.g. `https://login.microsoftonline.com/<DirectoryID>/oauth2/token`
`clientID`	Required: Client ID of the Azure Active Directory application. Possible values: String containing the client ID, e.g. `10925xxx-1111-4444-bd3d-82165c0d3841`
`clientSecret`	Required: Client secret of the Azure Active Directory application. Possible values: String containing the secret value.
`accountName`	Required: Fully qualified Data Lake Storage account name (FQDN). Possible values: String containing the FQDN, e.g. `<your_account_name>.azuredatalakestore.net`
`fileName`	File name This parameter is required for the operations `AppendFile`, `ConcatenateFiles`, `DeleteFile`, `ReadFile`, `RenameFile` and `UploadFile`. This parameter is optional for the operations `GetMetadata` and `SetPermission` Possible values: String containing the file name, e.g. `inputFile.txt` or `root/subfolder/inputFile.txt` The string may contain both the complete file path (`root/subfolder/inputFile.txt`) or the `fileName` (`inputFile.txt`) and `folderPath` (`root/subfolder`).
`folderPath`	Folder path; It can start and end with a slash. Possible values: `/`: Root folder Any other valid path, e.g. `root/subFolder/.../lastFolder`
`overwrite`	Flag to overwrite existing files. This parameter is only valid for the operations `UploadFile` and `RenameFile`. Possible values: `true`: Overwrite the existing file (defined in `fileName`) with the new content `false`: For the operation `UploadFile` the adapter returns an error, if the file already exists. For the operation `RenameFile` the adapter returns the status `1` but doesn't rename the source file.
`listFilesToConcatenate`	Name of the files to be concatenated (separate by colon). This parameter is required for the operation `ConcatenateFiles`. Possible values: Existing files (full path names) separate by colon, e.g. `files/File1.txt:files/File2.txt:otherPath/File3.txt`
`renamedObject`	New path name of the file or directory to be renamed. This parameter is required for the operations `RenameFile` and `RenameFolder`. Possible values: Path name, e.g. `files/newFileName.txt`
`octalPermissions`	Permissions to be set in unix octal form. For more information on the Octal Permission Representation, see also https://www.tutorialspoint.com/unix/unix-file-permission.htm. Example: `770` → `7` Own user with permission to read, write and to execute; `7` Group with permission to read, write and to execute; `0` Others with no permissions. This parameter is required for the operation `SetPermission`.
`maxEntriesToRetrieve`	Maximum number of entries to be retrieved. This parameter is optional and will be ignored, if the parameter `entriesRecursively` is enabled. It is useful for the operation `EnumerateDirectory`. Note that the server can limit the number of entries retrieved to a number smaller than the number specified. Possible values: Integer value greater than `0`.
`startAfterPathName`	The file or directory name after which to begin the enumeration. This parameter is optional and will be ignored, if the parameter `entriesRecursively` is enabled. It is useful for the operation `EnumerateDirectory`. Possible values: String containing the path name of the file or folder.
`entriesRecursively`	Enables to extract all the entries recursively. If the parameter is enabled `maxEntriesToRetrieve` and `startAfterPathName` are ignored. This parameter is optional and useful for the operation `EnumerateDirectory`. Possible values: `true`: All entries of the given folder path will be retrieved recursively. `false`: All entries of the given folder path will be retrieved, so only direct children will be retrieved.

Status values

`1`	The operation was executed successfully
`-1`	An error occurred during execution

Input

The adapter expects any input document for the operations UploadFile and AppendFile. The other operations do not require any input.

Output

Depending on the selected operation, the adapter will output various documents:

AppendFile

If the operation was executed successfully (status 1), the adapter outputs the input document.
In case of an error (status -1 ), the adapter outputs an XML document with the error description, e.g.

CreateFolder

If the operation was executed successfully (status 1), the adapter outputs the input document in case any input document was assigned.
In case of an error (status -1 ), the adapter outputs an XML document with the error description, e.g.

ConcatenateFiles

If the operation was executed successfully (status 1), the adapter outputs the input document.
In case of an error (status -1 ), the adapter outputs an XML document with the error description, e.g.

DeleteFile

If the operation was executed successfully (status 1), the adapter outputs the input document in case any input document was assigned.
In case of an error (status -1 ), the adapter outputs an XML document with the error description, e.g.

DeleteFolder

If the operation was executed successfully (status 1), the adapter outputs the input document in case any input document was assigned.
In case of an error (status -1 ), the adapter outputs an XML document with the error description, e.g.

EnumerateDirectory

If the operation was executed successfully (status 1), the adapter outputs an XML document containing a list with all files and folders found, e.g.:
Each Entry node corresponds to a file folder. The following attributes characterize each Entry:
- LastAccessTime: Time of the last access to the file or folder.
- LastModifiedTime: Time of the last modification.
- Group: ID of the group owning the file or folder.
- User: ID of the user owning the file or folder.
- Permission: Unix-style permissions string for this file or folder.
- Name: Name of the file or folder.
- FullName: Complete path of the file or folder.
In case of an error (status -1 ), the adapter outputs an XML document with the error description, e.g.:

GetMetadata

If the operation was executed successfully (status 1), the adapter outputs an XML document containing the metadata information of the file or directory.
The root element Metadata contains an Entry element with the following attributes:
- Group: ID of the group owning the file or folder.
- User: ID of the user owning the file or folder.
- Permission: Unix-style permissions string for this file or folder.
- Name: Name of the file or folder.
- FullName: Complete path of the file or folder.
- LastAccessTime: Time of the last access to the file or folder.
- LastModifiedTime: Time of the last modification or folder.
In case of an error (status -1), the adapter outputs an XML document with the error description, e.g.:

ReadFile

If the operation was executed successfully (status 1), the adapter outputs the file that was read:
In case of an error (status -1), the adapter outputs an XML document with the error description, e.g.:

RenameFile

If the operation was executed successfully (status 1), the adapter returns the input document in case any input document was assigned.
In case of an error (status -1), the adapter outputs an XML document with the error description, e.g.:

RenameFolder

If the operation was executed successfully (status 1), the adapter returns the input document in case any input document was assigned.
In case of an error (status -1), the adapter outputs an XML document with the error description, e.g.:

SetPermission

If the operation was executed successfully (status 1), the adapter returns the input document in case any input document was assigned.
In case of an error (status -1), the adapter outputs an XML document with the error description, e.g.:

UploadFile

If the operation was executed successfully (status 1), the adapter outputs the file that was uploaded.
In case of an error (status -1), the adapter outputs an XML document with the error description, e.g.:

If the folderPath parameter is empty, the file will be created in the account's root folder. If the path set in the folderPath parameter does not exist, it will be created.