What's the difference in heat rate of a CCGT power station at 50% load vs 100% load? What's the difference in heat rate of a OCGT at 50% load vs a CCGT at 100% load? What's the difference in heat rate between a inefficient peaking liquid fuel turbine or reciprocating engine to a CCGT at 100% load? To answer these questions, heat rate at least efficient operation will be more than double that at most efficient. These are where the real benefits of V2G come into play - removing peaks and troughs and allowing generation to operate in an efficient way. 50% reduction in fuel required for the same generation certainly makes the few percent conversion losses pale into significance. And this is before even getting into the mitigation of generating capacity (capital investment), which means power charge per kWh reduces.
To your point on grid efficiency. What's the difference in copper losses in a transformer (indicative of the transmission network) between 50% load and 100% load? Transmission and distribution efficiency is not linear but rather follows a curve. Iron losses will dominate at very low loadings, and copper losses (which vary proportional to the square of the current) will dominate at periods of high loadings. The grid is most efficient somewhere between the two. Using V2G to avoid pushing the grid into higher peak loadings on a summer day will make it slightly more efficient on average.
You mentioned that your neighbor would not be using the power you push out from V2G and that your power will need to go through 2 transformers at minimum. That is likely an incorrect statement. If you are on a shared transformer (which all urban customers and a lot of rural customers are) then your V2G 'generation' will almost certainly be consumed downstream of the distribution transformer and therefore have almost no network losses. A lot of it may even be consumed in your own house. Keep in mind that V2G will be likely occurring far after the solar peak, so you'll be offsetting the PV generation that has reduced as the sun went down.