Since heavier gases exert larger effects on rotordynamic stability, stability evaluation is important in developing or designing high-pressure compressors. To evaluate the rotor stability during operation, an excitation test using a magnetic bearing is the most practical method. In stability analysis, labyrinth seals can produce significant cross-coupling forces, which particularly reduce the damping ratio of the first forward mode. Therefore, forward modes should be distinguished from backward modes in the excitation test. One method that excites only the forward modes, not the backward modes (and vice versa), is the use of a rotating excitation. In this method, the force is simultaneously applied to two axes to excite the rotor in circular orbits. Two trigonometric functions, i.e., cosine and sine functions, are used to generate this rotation force. Another method is the use of a unidirectional excitation and a mathematical operation to distinguish the forward whirl from the backward whirl. In this method, a directional frequency response function that separates the two modes in the frequency domain is obtained from four frequency response functions by using a complex number expression for the rotor motion. In this study, the latter method was employed to evaluate the rotor stability of a high-pressure compressor. To obtain the frequencies and damping ratios of the eigenvalues, the curve fitting based on system identification methods, such as the prediction error method, was introduced for the derived frequency response functions. Firstly, these methods were applied to a base evaluation under a low-pressure gas operation, in which the stability mainly depends on the bearing property. Using the obtained results, the bearing coefficients were estimated. Next, the same methods were applied to stability evaluations under high-pressure gas operations. The destabilizing forces were also estimated from the test results and compared with the calculation results.