Here is a red herring to add onto this mix...
It is most likely that, that is the cause. I was going to mention it, say, if the tool’s center is moving at such a slower pace, then the outside will fail to cut as well just like in the case of ballnose cutter. Moving further outside, the speed will be higher and consistent hence it will efficiently cut with a more sensible speed and less force, hence a little deflection.
It is much easier to work when using a 90o tool. The tool is at zero if the depth is equal to the offset. For example, when you add an offset of 3mm( using the allowance box in V-carve, or simply select 6mm cutter) and begin at Z= -3, then the tool is at zero.
If you cut down to Z= -5, you will get a 2mm chamfer. In the real sense, this is the easiest way to begin and just lower the z-axis until when the chamfer looks nice as that is what generally matter more than absolute value.
You should be on the look for what the other side of the V-cutter is doing. This is because too much offset and nested parts will lead to cutting adjacent parts too much. I use a standard 90o ½ inch router bit for my chamfer. What cutter did you use? IDK what your measurements were but I suppose they will be much much similar.