Proton conductivity, structural and morphological characteristics of new polymer containing composite membranes based on CsH2PO4 and ultra-dispersed polytetrafluoroethylene (UPTFE) have been investigated. Optimization of the preparation method of membranes was carried out. The CsH2PO4 (P2(1)/m) phase was shown to persist in the UPTFE polymer matrix in a whole range of compositions. The proton conductivity of (1 - x)CsH2PO4 - xUPTFE (x = 0.035-0.3 x-weight fraction) electrolytes with various composition was studied as a function of temperature and partial pressure of water vapor. The conductivity values were close to the pure CsH2PO4 at x <0.1. With x increase the conductivity decreased due to the "conductor-insulator" percolation effect within one order of magnitude. The unit cell parameters of CsH2PO4 in polymer matrix were calculated and microstructure of electrolytes was studied.