Why would a Lyft driver just drive off?

This past Saturday night, I set up a scheduled ride (e.g. I didn't just open the app and request a ride) on Lyft from a very close in Virginia location within site of downtown Washington, D.C. out to a small town in the outer reaches of close in suburban VA county. Drive would have been about 45 minutes long and definitely AWAY from the "city" (Washington) area out to the 'burbs.

I got the notification that driver X was on his way for the scheduled pickup and, then, that he arrived. My traveling group was standing at the driveway waiting for him as he pulled up, and I had my back to the street locking the house door. In the time I turned around, the driver drove off. We were thinking that he was turning the car around at the end of our block (about 100-200 feet down the road) because it would point him in the right direction to leave our neighborhood and head to the destination.

Nope. He kept going. Never to return.

In less time that it took for us to realize driver X wasn't coming back, I looked down at my Lyft app (thinking I needed to press something to say in substance, "here, waiting ..." the app assigned us to another driver 20 minutes away who eventually came and took us to our destination.

[Side note]. Friends of ours took a car from about the same location (about 10 blocks away) out to the same destination in the outer reaches of close in suburban VA county (we were meeting for dinner, FWIW). During back and forth chit chat about the eventual return trip their driver said that, on a Saturday night, NO driver wanted to be driving out to the suburbs. They all want to be working close in to the City. Didn't say why. But, in the post mortem, we figured (a) they can complete more trips - however short which add up to some "goal" of completing X number of rides per day (b) with fuel prices nowadays and the added Lyft "fuel surcharge" per ride, they can keep banking a lot of short rides and repeated "fuel surcharges" per ride.

I thought it might be that as driver X was pulling up, he realized where the ride was about to take him and he just bailed - put something into the driver app to "canel" the ride. I didn't get a notification of a cancelled ride or that driver X rejected me for some reason. Just trying to figure out why and whether it changes my "rating" with Lyft.