Skip to content

GH-55683: [C++][FS][Azure] Use the DFS URL for the Data Lake client with SAS credentials - #55684

Open
LigaData-ltsentsiper wants to merge 1 commit into
apache:mainfrom
LigaData-ltsentsiper:GH-55683-azure-sas-dfs-url
Open

LigaData-ltsentsiper wants to merge 1 commit into
apache:mainfrom
LigaData-ltsentsiper:GH-55683-azure-sas-dfs-url

Conversation

@LigaData-ltsentsiper

Copy link
Copy Markdown

Rationale for this change

With a SAS token, MakeDataLakeServiceClient builds the DataLakeServiceClient from the blob URL. The Azure SDK sends
Data Lake REST calls to the URL it is given, so the hierarchical namespace check goes to the blob endpoint, gets a 400, and
AzureFileSystem treats a hierarchical namespace account as flat: Move() is "not implemented" and DeleteFile() fails.

What changes are included in this PR?

MakeDataLakeServiceClient uses AccountDfsUrl(account_name) + sas_token_ for the SAS credential kind, like every other
credential kind.

Are these changes tested?

Manually, against a real storage account with hierarchical namespace: a small program calling AzureFileSystem directly
(CreateDir, write, Move(), GetFileInfo, DeleteFile, results in the issue) with stock and patched 26.0.1 and an
account SAS, and an application that writes Parquet files and renames them (2,000,000 rows) with an account SAS and a
container SAS. No automated test: the existing Azure tests run against Azurite, which has no hierarchical namespace.
A unit test in TestAzureOptions could build AzureOptions with a SAS token and check that
MakeDataLakeServiceClient()->GetUrl() uses the dfs host; I have not written or run it, and can add it if wanted.

Are there any user-facing changes?

SAS token users on hierarchical namespace accounts get the hierarchical namespace code paths (Move(), DeleteFile(),
CreateDir) instead of the flat namespace ones.

🤖 Generated with Claude Code

…ient with SAS credentials

With a SAS token, MakeDataLakeServiceClient built the DataLakeServiceClient from the blob URL. The Azure SDK sends
Data Lake REST calls to the URL it is given, so the hierarchical namespace check went to the blob endpoint, got a 400
and AzureFileSystem treated a hierarchical namespace account as flat (Move() not implemented, DeleteFile() failing).
Use AccountDfsUrl like every other credential kind.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Python][C++][FS][Azure] AzureFileSystem with a SAS token treats a hierarchical namespace account as flat (Move() not implemented, DeleteFile fails)

1 participant