Honestly, both are polite and correct. I would use "nice speaking with you" over the phone and "nice speaking to you" in person. Because over the phone your speaking with something. In person your talking directly to someone.
Chat with our AI personalities