This paper reports on the results of an experiment to test the use of a Peer Instruction (PI) pedagogical model in a small class, high school environment. The study reports findings based on a population of 213 high school students attending algebra based physics courses, both Honors and A level, taught by 5 different instructors. The results show a correlation between use of Peer Instruction and improved student conceptual understanding, as demonstrated by gains on a pre-/post- assessment instrument (FCI). However, there also appears to be a number of other factors that strongly influence the resulting gains. In addition to instructor differences, the data seem to indicate that students who are more "physics-inclined" and can answer questions correctly prior to instruction and prior to any Peer Instruction discussion subsequently achieve higher gains as measured by the FCI. While this is to be expected, the use of normalized gains is intended to mitigate this result, but it appears to be prevalent nonetheless. This raises questions as to what degree the FCI gains can be attributed to the use of Peer Instruction, to teacher differences, to student ability level or to simply increased familiarity with the question types presented on the FCI.