Influenced by detectors' material, manufacturing technology etc, every detector in infrared focal plane array (IRFPA) will output different voltages even if their input radiation flux is the same. And this is called non-uniformity of IRFPA. At the same time, the high background temperature, low temperature difference between targets and background and the low responsivity of IRFPA result in low contrast of infrared images. So non-uniformity correction and image enhancement are important techniques for IRFPA imaging system. This paper proposes a new real-time infrared image processing system based on Field Programmable Gate Array(FPGA). The system implements non-uniformity correction, image enhancement and video synthesization etc. By using parallel architecture and pipeline technique, the system processing speed is as high as 50Mx12bits per second. It is appropriate greatly to a large IRFPA and a high frame frequency IRFPA imaging system. The system is miniatured in one FPGA.