Test method, test content and test type are supposed to affect test taker's performance and have close connectionswith test characteristics. However, these issues have not been subject to much in-depth investigations. To shed somelights on the issues like test item arrangements in relation to test taker's performance, test usefulness criteriaincluding test Validity, Reliability, Impact, Interactiveness, Authenticity, and Practicality, and also test characteristicssuch as Item Facility (IF), Item Discrimination (ID), and Choice Distribution (CD), this ex post facto design studywas conducted with a group of Iranian EFL learners. For this purpose, some university randomly selected studentsmajoring in English Language Teaching, English Translation Studies, and English Literature and learners fromdifferent language institutes received a version of the Nelson proficiency test. Then, two different versions of a sameresearcher-made test in terms of item arrangements were administered to 116 students obtaining an acceptable scorein the Nelson proficiency test. Simultaneously, a Test Usefulness Criteria Questionnaire developed and validated byAbbasian and Nassirian (2015) based on Bachman and Palmer’s (1996) framework, was also attempted by theparticipants. Respective statistical analyses revealed contradictorily that test item arrangement did not have anysignificant effect on the performance of the test takers. However, test usefulness criteria, though with varying extent,proved to be subject to test item arrangement. In the same vein, IF and ID were also affected in the light of testmethod facet. The first contradictory finding leaves room open for further research, while the reaming findings offerinsights to test developers, classroom teachers and all practitioners to pay due attention to test method facets in theireducational assessment decisions as the learners' performance, nature of the measurement devices as well as thenature of the construct itself are all affected test method factors.