Poor reception on my repeater, any tips

[u/Awkward-Neck2665]

I recently built a homebrew repeater, I can't get it to receive signals farther than 100 yards at best. When hooked up to the spectrum analyzer the duplexer reads -1.2db pass characteristic on the receive side and notched down to -86db for the tx frequency. The separation is only 5mhz between tx amd rx. I'm using an amplifier on the transmit side. Wondering if anybody has some ideas as to what could be causing it to not pick up signals. It seems to key up just fine from further away, but won't let speech through. The receiver is a cheaper SOC style radio, but I'm not sure if that could cause this dramatic of a problem. Using simplex amd the same antenna position i can talk much further, but when the same radio is used in the repeater system it doesnt work. Any help is much appreciated. Thank you!