That looks like a misunderstanding to me. You have to connect somewhere to access the bucket. I do not think that "cross-account" buckets are anyhow exceptional in this respect. So what exact problem do you have when trying to access the bucket?

More info here:
https://repost.aws/knowledge-center/cross-account-access-s3
and here:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-walkthroughs-managing-access-example2.html

Basically, WinSCP should allow S3 URLs of type s3://EXAMPLE-BUCKET instead of forcing https://EXAMPLE-BUCKET.s3.amazonaws.com or https://s3.amazonaws.com/EXAMPLE-BUCKET.
This will allow accessing buckets with cross-account delegation.