OP - you may have read my posts about using a proximity sensor for an index signal rather than the built-in Z signal on my encoder.
I did this because the encoder is on my motor and the pulley ratio between motor and spindle is not 1:1.
I had to have a spindle index signal, so I installed a proxy & trigger on the spindle.
I'm using a 200 line (800CPR) encoder on my motor so as not to overwhelm the BOB and UC400ETH. As others have pointed out, your 1024 encoder may have signals too short for reliable operation.
I suggest replacing the encoder with a lower count one which should provide a longer signal - both for A/B and for a Z signal.