STS-134

Results are given in the format Speed test city, <Hosted by organization>: DL speed/UL speed/RTT (ping) time. All tests were done with Ookla Speedtest (they have a mobile app and they also run the site at speedtest.net) and were conducted in Shanghai at around 9am, so as to avoid the evening rush when everyone starts using VPNs in China and when results can be more affected by how many simultaneous users there are on the paid VPNs.

1. China Unicom (Mainland SIM Card). This card has a Chengdu IP address.
Although this card is 4G capable, my Nexus 5 can't use the 4G network, and so the actual tests were done on China Unicom 3G. Tests were done without a VPN, and then with VPNs to various locations.

No VPN
Chengdu: 1.98M/1.96M/87ms
Not sure what's up with this, maybe a lack of bandwidth at the Chengdu endpoint for Ookla Speedtest?

Express VPN in Hong Kong
Hong Kong, Website Solution Limited: 2.21M/1.73M/224ms
Hong Kong, China Broadband Communications: 0.78M/2.84M/229ms

Express VPN in Santa Clara. Traffic exits at Delaware.
Philadelphia, PA: 1.66M/1.75M/389ms

Express VPN in San Jose, CA
San Jose, CA, Sonic.net: 2.10M/1.78M/299ms
Palo Alto, CA, Singtel: 1.89M/1.76M/308ms

VPN.AC in San Jose, CA
San Jose, CA, Sonic.net: 1.85M/1.81M/327ms
Palo Alto, CA, Singtel: 1.13M/1.33M/474ms

VPN.AC in Los Angeles, CA
Los Angeles, CA, FPT Telecom: 2.18M/1.81M/283ms

VPN.AC in Hong Kong
Hong Kong, Website Solution Limited: 1.77M/1.75M/158ms
Hong Kong, China Broadband Communications: 1.46M/1.79M/146ms

Shadowsocks (private server in my living room in Silicon Valley)
San Jose, CA, Sonic.net: 2.20M/2.20M/286ms
Palo Alto, CA, Singtel: 2.38M/2.13M/279ms

OpenVPN (private server in my living room in Silicon Valley)
San Jose, CA, Sonic.net: 2.13M/1.80M/285ms
Palo Alto, CA, Singtel: 2.08M/1.63M/282ms

2. China Unicom Hong Kong Cross Border SIM. This card has a Hong Kong IP address
This card is not 4G capable. It only allows access to China Unicom's 3G network, which my phone, a Nexus 5, is capable of using.

Hong Kong, Website Solutions Limited: 8.38M/2.10M/104ms
Hong Kong, China Broadband Communications: 5.82M/2.11M/180ms
San Jose, CA, Sonic.net: 4.79M/1.74M/249ms
Palo Alto, CA, Singtel: 4.34M/2.19M/244ms

3. AT&T SIM. Card selected China Mobile (phone only gets EDGE) and was using an IP address from Vacaville, CA
San Jose, CA, Sonic.net: 0.18M/0.06M/522ms
Palo Alto, CA, Singtel: 0.18M/0.04M/514ms
San Francisco, CA, AT&T: 0.13M/0.08M/563ms

4. AT&T SIM. China Unicom roaming forced for 3G. Card was using an Atlanta, GA IP address
In theory, this card should be able to get 4G service as well, but my Nexus 5 can't use Unicom's 4G network (see comment for #1 above as well).
Atlanta, GA: 1.51M/1.17M/377ms
San Jose, CA, Sonic.net: 444ms ping time. Throughput test did not finish.
Palo Alto, CA, Singtel: 0.26M/1.47M/438ms

I think these results are pretty self explanatory, aren't they? If you are visiting China, do not buy a mainland SIM card. Do not use a SIM card from your home country. Buy the China Unicom Hong Kong cross border SIM card and use that.

Interesting notes from the results:
- My private Shadowsocks and OpenVPN server beat both ExpressVPN's and VPN.ac's Silicon Valley endpoints slightly.
- If all you want to do is access services like Google and Youtube, you'd probably want to look at the Hong Kong numbers, because Google does have servers in Hong Kong, and the results there are even more skewed in favor of the cross border SIM. Both download results were far in excess of 5Mbps with ping times of around 104-180ms.
- VPN.ac had the best ping times to Hong Kong from its Hong Kong endpoint, but the throughput bandwidth was lacking. Perhaps traffic from the cross border SIM cards has some sort of priority in passing through the Chinese firewall?
- The AT&T SIM card results were awful when the phone was on China Mobile. But that was pretty much expected.
- The AT&T SIM card results were okay when China Unicom roaming was forced so that my phones got 3G instead of EDGE, about on par with the mainland SIM using VPN.ac's Silicon Valley endpoint. So when forcing China Unicom roaming, an AT&T SIM can be more reliable than using a paid VPN, because it "just works", although it is a very expensive way to get that reliability (US$120 for 800MB). And given that I still recommend having a VPN so that you can use wifi, I think it is unnecessary for most people.

Pitfalls:
- The CUHK Cross Border Dual Extra SIM comes with HK$80 of stored value, and a HK$18 monthly service fee is immediately charged to the card on activation, leaving you with HK$62. If you want to order the HK$118/1GB plan, you'll therefore need to top up the card after you activate the SIM but before you subscribe to data. You'll need an alternate data connection in order to do this online.
- My bank declined the transaction through Verified by Visa multiple times. I had to call them to get the transaction to go through.

jackal

Awesome--thanks for the detailed testing and report. You confirmed what I suspected (and recommended what I ended up doing on both trips).

While I didn't have a local mainland SIM to test with and didn't want to spend the coin to test AT&T roaming, my experiences with mainland wired connections (ones that would get 100mbps locally within mainland China but would slow to a barely-1-mbps-with-noticeable-packet-loss as soon as I tried to penetrate the Great Firewall) left me suspecting that mainland SIMs would likely have similar [lack of] reliability.

I had nothing but great experiences with the Unicom Cross-Border SIM, and although it's a bit more expensive than using a mainland SIM, the quality of service is excellent. I'll continue to do that, then, thanks to your post here!

STS-134

I don't think it was all that obvious before the tests. It could be due to a lack of bandwidth between Mainland China and the outside, in which case it would affect all data connections to the outside, HK SIMs included. But the HK SIM traffic does seem to be getting priority treatment.

Also, Android is broken. It constantly tells me that it has no internet connection with the mainland SIM, even after I turn on the VPN. It sees every wifi network I connect to as having no internet access, and therefore does not use the wifi unless I explicitly tell it to "use the network as is". It never reconnects to wifi networks automatically due to seeing it as having no internet access.

At least with the HK SIM the annoying "!" that means no internet access doesn't show up on the cellular side.

Anyway, time to get on Google's case about this. I have the latest Android version (6.0.1 Marshmallow).

jamar

Most likely explanation for the ! is that on Android for devices sold outside China, it pings Google to determine whether it has a connection to the outside or not. Since China's blocking Google, any mainland-based connection will give you the ! while the HK SIM won't.

STS-134

Yes, I understand the problem they were trying to solve (users of past Android versions would connect to wifi, the wifi wouldn't have an internet connection, and they'd open a browser and wouldn't be able to do anything, and they wouldn't realize that they had to turn the wifi off or disconnect from the AP to get packets to route over cellular again).

But when they do this without allowing the user control over the feature, they also create additional problems. For example:

- I can't tell whether I have internet connectivity on either wifi or cellular with the mainland SIM card

- With the HK SIM, it will connect to wifi and use that connection if I tell it to "Use the network as is" after connecting. But if I go out of range and come back, it won't reconnect automatically. If I force it to reconnect, there's no longer an option to tell it to "Use the network as is"; this only shows up after connecting the first time. In this case, the only way I can get it to actually route packets over wifi is to use the data kill switch under Settings->More->Cellular networks->Data Roaming, and set "Connect to data services when roaming" to OFF, or to put the phone in airplane mode.

You tell me whether you think the average person would ever be able to figure this out, let alone find out how to manipulate the system to route packets properly, ESPECIALLY because Android does not give you an active indication that it's routing packets over cellular when it's roaming; the "E", "3G", "H", and "LTE" icons are simply replaced by an "R" that shows up whether you have data connectivity or not. I think the average user would be likely to connect to wifi, not verify how the packets are being routed, and run up a huge bill on their SIM card (which, by the way, could also be a US SIM on international roaming, because the phone would see the SIM as having "internet access" while the wifi would not have "internet access", which would be a VERY expensive mistake).

At least with the old system, it was clear that there was a problem (internet doesn't work) and that something needed to be done about it (turn wifi off). With the new system, it'll route packets in unexpected ways and run up big bills in the process if you aren't careful.

Oh, and there's another annoyance with Marshmallow. The "Network Operators" menu item under Settings->More->Cellular networks has disappeared with I have my AT&T SIM in the phone; it was there in previous versions of Android. This prevents me from actively forcing the phone to use China Unicom, so it'll hop onto China Mobile at times, which means that data gets painfully slow (China Mobile uses TD-SCDMA for 3G which the Nexus 5 does not support).

STS-134

Well something very interesting happened today when I went to the Shanghai Disney Resort. My Nexus 5 with the China Unicom Mainland SIM started showing LTE! I'm not sure what's going on with the network there, but I suspect that Unicom expects a lot of international visitors with phones that can't use certain LTE bands and may have modified their network to allow more devices to use LTE, which would help alleviate an expected bandwidth crunch.

I ran a test with VPN on (VPN.ac in Hong Kong) and ran the test to a HK endpoint, and got 1.43M/0.17M/152ms. Then I switched off the VPN and ran a test to a Shanghai endpoint, and got 34.28M/12.93M/99ms. Wow! But those speeds apparently only exist for accessing stuff within China. The HK SIM got 15.88M/1.35M/77ms from the same spot in Disneytown which is far above what the VPN + mainland SIM got.