The Session.RemoveFiles removes a file/files that you give it in argument.
So Session.RemoveFiles("/path/file") removes just the one file.

https://winscp.net/eng/docs/library_session_removefiles

I am able to get a list of all files on the server and able to download all of them to a local directory. Now I need to delete but ONLY ONE file out of (x) files on the server. I saw session.removefiles but it looks like that will remove ALL files.

Knowing the file name, Need only to remove 1 file.