Abstract:The analytical expression of bit error probability in a balanced differential phase-shift keying (DPSK) optical receiver considering nonlinear phase noise and EDFA ASE noise is given, which is very useful to estimate the performance of DPSK balanced and unbalanced receiver in optical communication system. Through analysis, if only nonlinear phase noise is considered, both the balance and unbalanced receivers have the same performances. But if adding the ASE noise of EDFA, the balanced receiver is better. This work has been supported by the National Natural Science Foundation of China (No.60772013) and the Fund of Huazhong Uiversity of Science and Technology (2006M041)