Motivated by recent developments in solid electrolytes based on lithium phosphorous oxynitride (LiPON) materials, we carried out computer simulations in order to develop an understanding of the detailed mechanisms of ionic transport. Starting with crystalline Li3PO4, we investigated some possible structures of isolated defects associated with extrinsic Li ion vacancies and interstitials. In particular, we identified a "PNP" structure which involves a bridging N between two phosphate groups, formed from the combination of an O vacancy and a N substitution, stabilizing a Li ion vacancy. We also studied the effects of N substituting for a tetrahedral O in a phosphate group which stabilizes a Li interstitial ion. In addition to studying the structures of these isolated defects as sources of extrinsic mobile Li ions, we also modeled their effects on the mobilities of the ions.