GitLab CI: Unable to download job artifact from another project – CI_JOB_TOKEN returns 404, private token returns 401

I’m facing an issue while trying to download a job artifact from one GitLab project into another using GitLab CI/CD.

What I’m trying to achieve

In the project A, I want to improve CI testing by:

Downloading and executing an SQL artifact generated by another project B id:123 ( PROJECT B id:123)

This setup works correctly on my local machine, but fails in GitLab CI.

Setup details

Source project: (PROJECT B id:123)

Consumer project: project A
Artifact URL:
https://gitlab.example.com/api/v4/projects/608/jobs/artifacts/main/raw/gs_db.sql?job=generate_sql_file id/jobs/artifacts/main/raw/test_db.sql?job=testsmangosql

What I have tested

1. Using CI_JOB_TOKEN (recommended approach)

Result:404 Project Not Found

Observations:

The project 1 is added to the CI/CD job token allowlist in Project B ID-123

Added at both group level and explicit project level

Job token permissions are configured via CI/CD settings

Despite this, GitLab still returns 404, which usually indicates that the job token cannot access the target project or artifact endpoint

2. Using a Private Access Token
401 Unauthorized
Observations:

This suggests the token itself is valid

However, it may not be accessible in the CI pipeline context (e.g. protected variable or branch restriction)

Additional validation steps

Created a separate pipeline stage dedicated only to artifact download

Tested multiple download methods, including the official GitLab API approach:
https://archives.docs.gitlab.com/18.6/api/job_artifacts/#download-a-single-artifact-file-by-reference-name

Attempted to manually download the artifact via the GitLab UI “Download artifact” button

This also results in 401 / 404 errors

The same behavior is consistently observed

Local download using a personal access token works correctly

Questions

Are there any additional CI_JOB_TOKEN permissions or scopes required to download job artifacts from another project?

Are there any instance-level or project-level restrictions that could cause 404 Project Not Found even when the project exists and is allowlisted?

Does the job artifact download endpoint require permissions beyond what is configured in CI/CD → Job token access?

Any guidance or clarification would be appreciated.