So did you try using a hostname with the format my-bucket-ap-123456789012.s3-accesspoint.us-west-2.amazonaws.com? As per:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-access-points.html

We are currently accessing clients' S3 buckets via S3 Access Points: https://aws.amazon.com/s3/features/access-points/ and they use ARN as the bucket like: arn:aws:s3:region:account-id:accesspoint/access-point-name. Does the S3 explorer support the setup for the access points via IAM role which has a full s3 permission policy attached?