Future large visual information systems (such as image databases and video servers) require effective and efficient methods for indexing, accessing, and manipulating images based on visual content. This paper focuses on automatic extraction of low-level visual features such as texture, color, and shape. Continuing our prior work in compressed video manipulation, we also propose to explore the possibility of deriving visual features directly from the compressed domain, such as the DCT and wavelet transform domain. By stressing at the low-level features, we hope to achieve generic techniques applicable to general applications. By exploring the compressed-domain content extractability, we hope to reduce the computational complexity. We also propose a quad-tree based data structure to bind various signal features. Integrated feature maps are proposed to improve the overall effectiveness of the feature-based image query system. Current technical progress and system prototypes are also described. Part of the prototype work has been integrated into the Multimedia/VOD testbed in the Advanced Image Lab of Columbia University.