Home | Research Experience | Selected Publications | Research Grants |
Research Recognition |
Teaching & Student Projects | Major Systems for Industries
Major Presentations Employment History | Other Professional Activities
 
 
 
 
 
 
Selected Recent Publications

Perceptual Image Quality Assessment
...we are able to build machines that perform significantly better and quicker than many of our organs, like arms and legs; however, when it comes to model our eyes and brain toward human perception, the odyssey proves to be much more difficult.

K. Gu, J. Zhou, J. F. Qiao, G. T. Zhai, W. Lin, A. C. Bovik, “No-reference quality assessment of screen content pictures”, IEEE Transactions on Image Processing,  26(8): 4005-4018, 2017
T. J. Liu, K. H. Liu, J. Y. Lin, W. Lin, C.-C. Jay Kuo, "A ParaBoost Method to Image Quality Assessment", IEEE Transactions on Neural Networks and Learning Systems,  28(1): 107 - 121, 2017
S. G. Wang, C. Deng, W. Lin, G. Huang, B. Zhao, “NMF-based Image Quality Assessment Using Extreme Learning Machine”, IEEE Transactions on Cybernetics,  47(1): 232 - 243, 2017
Y. Zhang, Y. Fang, W. Lin, X. Zhang, L. Li, “Backward Registration Based Aspect Ratio Similarity (ARS) for Image Retargeting Quality Assessment”, IEEE Transactions on Image Processing, 25(9):4286- 4297, 2016
H. Yang, Y. Fang, W. Lin, “Perceptual Quality Assessment of Screen Content Images”, IEEE Transactions on Image Processing, accepted, 2015. (database)
T-J Liu, W. Lin, C.-C. Jay Kuo, "A ParaBoost Method to Image Quality Assessment", IEEE Transactions on Neural Networks and Learning Systems, 2015, accepted.
Y. Zhang, Y. Fang, W. Lin, X. Zhang, L. Li, “Backward Registration Based Aspect Ratio Similarity (ARS) for Image Retargeting Quality Assessment”, IEEE Transactions on Image Processing, accepted. (Source Codes)
J. Wu, W. Lin, G. Shi, Y. Zhang, “Visual Orientation Selectivity based Structure Description”, IEEE Transactions on Image Processing, accepted, 2015.
F. Shao, K. Li, W. Lin, G. Jiang,  M. Yu, Q. Dai, “Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties”, IEEE Transactions on Image Processing, VOL. 24, NO. 10, pp. 2971 – 2983, 2015.
L. Dong, Y. Fang, W. Lin, H. S. Seah, “Perceptual Quality Assessment for 3D Triangular Mesh based on Curvature”,  IEEE Trans. Multimedia, accepted, 2015.
F. Shao, W. Lin, S. Wang, G. Jiang,  M. Yu, Q. Dai, “Learning receptive fields and quality lookups for blind quality assessment of stereoscopic images”, IEEE Transactions on Cybernetics, 2015, accepted.
J. Wu, W. Lin, G. Shi, L. Li, “Orientation Selectivity based Visual Pattern for Reduced-Reference Image Quality Assessment”, Visual Acuity Inspired Saliency Detection by Using Sparse Features”, Information Sciences, accepted.
K. Gu, G. Zhai, W. Lin, X. Yang, W. Zhang, “Learning a Blind Quality Evaluation Engine of Screen Content Images”, Neurocomputing, aacepted
Q. Li, W. Lin, Y. Fang, “No-reference quality assessment for multiply-distorted images in gradient domain”, IEEE SIGNAL PROCESSING LETTERS, 23(4):541-545, 2016 (Source Codes)
F. Zhang, W. Jiang, F. Autrusseau, W. Lin, “Exploring V1 by modeling the perceptual quality of images”, Journal of Vision, VOL. 14, NO. 1, Article 26, 2014, (download)
F. Shao, S. Wang, W. Lin, G. Jiang,  M. Yu, “Blind image quality assessment for stereoscopic images using binocular guided quality lookup and visual codebook”, IEEE Trans.on Broadcasting, accepted.
Y. Fang, W. Lin, Z. Fang, Z. Chen, C-W Lin, C. Deng, “Visual Acuity Inspired Saliency Detection by Using Sparse Features”, Information Sciences, accepted.
L. Li, W. Lin, X. Wang, G. Yang, K. Bahrami, A. C. Kot, “No-Reference Image Blur Assessment Based on Discrete Orthogonal Moments”, IEEE Transactions on Cybernetics, accepted.
K. Gu, G. Zhai, X. Yang, W. Lin, W. Zhang, “No-Reference Image Sharpness Assessment in Autoregressive Parameter Space”, IEEE Transactions on Image Processing, accepted, 2015.
H. Yang, Y. Fang, Y. Yuan, W. Lin, “Subjective quality evaluation of compressed digital compound images”, Journal of Visual Communication and Image Representation, accepted.
Screen images are increasingly important, so is the evaluation for them.
Y. Fang, W. Lin, S. Winkler, “Review of Existing QoE Methodologies”, Multimedia Quality of Experience (QoE): Current Status and Future Requirements, eds. C. W. Chen, et al., John Wiley & Sons, 2015.
F. Shao, K. Li, W. Lin, G. Jiang, M. Yu, “Using Binocular Feature Rivalry for Blind Quality Assessment of Stereoscopic Images”, IEEE SIGNAL PROCESSING LETTERS,VOL. 22, NO. 10, pp. 1548 - 1551, 2015.
L. Li, Y. Zhou, J. Wu, W. Lin, H. Li, “GridSAR: Grid strength and regularity for robust evaluation of blocking artifacts in JPEG images”, Journal of Visual Communication and Image Representation, accepted.
S. Wang, K. Ma, H. Yeganeh, Z. Wang, W. Lin, “A Patch-Structure Representation Method for Quality Assessment of Contrast Changed Images”, IEEE SIGNAL PROCESSING LETTERS, 2015, accepted
A. Liu, W. Lin, H. Chen, P. Zhang, “Image Retargeting Quality Assessment Based on Support Vector Regression”, Signal Processing: Image Communication, VOL. 39, Part B, pp. 444–456, 2015
L. Li, Y. Zhou, W. Lin, J. Wu, X. Zhang, B. Chen, “No-reference quality assessment of deblocked images”, Neurocomputing, accepted
Y. Fang, K. Ma, Z. Wang, W. Lin, Z. Fang,  G. Zhai, “No-Reference Quality Assessment for Contrast-Distorted Images Based on Natural Scene Statistics”, IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 7, pp. 838 - 842, 2014, (download(Source Codes)
C-C Hsu, C-W Lin, Y. Fang, W. Lin, “Objective Quality Assessment for Image Retargeting Based on Perceptual Geometric Distortion and Information Loss”, IEEE Journal of Selected Topics in Signal Processing, VOL. 8, NO. 3, pp. 377-389, 2014, (download)
J. Wu, W. Lin, G. Shi, “Image Quality Assessment with Degradation on Spatial Structure”, IEEE SIGNAL PROCESSING LETTERS, VOL. 21, NO. 4, pp. 437 - 440, 2014, (download)
Y. Fang, K. Zeng, Z. Wang, W. Lin,  Z. Fang, C. Lin, “Objective Quality Assessment for Image Retargeting Based on Structural Similarity”, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, VOL. 4, NO. 1, pp. 95 - 105, 2014, (download)
L. Li, W. Lin, H. Zhu, “Learning Structural Regularity for Image Blockiness Evaluation”, IEEE SIGNAL PROCESSING LETTERS, VOL. 21, NO. 4, pp. 918 - 922, 2014, (download)
F. Zhang, W. Lin, Z. Chen, K. N. Ngan, “Additive Log-logistic Model for Networked Video Quality Assessment”, IEEE Transaction on Image Processing, VOL. 22, NO. 4, pp. 1536 - 1547, 2013 (download)

J. Wu, W. Lin, G. Shi, A. Liu, “A Perceptual Quality Metric with Internal Generative Mechanism”, IEEE Trans. on Image Processing, VOL. 22, NO. 1, pp. 43- 54, 2013 (download)
Our eyes distinguish orderly and disorderly visual signals, so does our quality metric built.
T-J Liu, W. Lin, and C.-C. Jay Kuo, “Image Quality Assessment Using Multi-Metric Fusion (MMF)”, IEEE Transaction on Image Processing, VOL. 22, NO. 5, pp. 1793-1807, 2013 (download)

T-J Liu, Y-C Lin, W. Lin, and C.-C. Jay Kuo, “Visual Quality Assessment: Recent Developments, Coding Applications and Future Trends”, APSIPA Transactions on Signal and Information Processing, Vol. 2, e4 (20 pages), 2013, (download)

J. Wu, W. Lin, G. Shi, A. Liu, “Reduced-Reference Image Quality Assessment with Visual Information Fidelity”, IEEE Trans. Multimedia, Vol. 15(7), pp. 1700-1704, 2013, (download)
L. Ma, C. Deng, K. N. Ngan, and W. Lin, “Recent Advances and Challenges of Visual Signal Quality Assessment”, China Communications, Vol. 10(5), pp. 62 - 78, 2013, (download)
F. Shao, W. Lin, S. Gu, G. Jiang, T. Srikanthan, “Perceptual full-reference quality assessment of stereoscopic images by considering binocular visual characteristics”, IEEE Transaction on Image Processing, Vol. 22, No. 5, pp. 1940-1953, 2013 (download)
M. Narwaria, and W. Lin, "SVD-Based Quality Metric for Image and Video Using Machine Learning”, IEEE Trans. on Systems, Man, and Cybernetics--Part B, Vol. 42(2), pp. 347 - 364, 2012 (download) (Source Codes)

Attempt toward a new paradigm for visual quality evaluation: the machine-learning approach to avoid unrealistic assumptions with feature pooling in the existing metrics


A. Liu, W. Lin, M. Narwaria, “Image Quality Assessment Based on Gradient Similarity”, IEEE Transaction on Image Processing, Vol. 21(4), pp. 1500 - 1512, 2012 (download(Source Codes)
The research defines the gradient similarity (GS) for better IQA, remedying a major drawback in prior work.

M. Narwaria, W. Lin, I. McLoughlin, S. Emmanue, L. T. Chia, “Fourier Transform Based Scalable Image Quality Measure”,  IEEE Trans. on Image Processing,    Vol. 21(8), pp.  3364 – 3377, 2012 (download) (Source Codes)
The use of phase and magnitude of FT in scalable visual quality evaluation

Q. Xu, Y. Yao, T. Jiang, Q. Huang, W. Lin, B. Yan, “HodgeRank on Random Graphs for Subjective Video Quality Assessment”, IEEE Trans. Multimedia, Vol. 14(3), pp.844 – 857, 2012 (download
To achieve efficient subjective assessment, without jeopardizing the assessment accuracy

J. Wu, W. Lin, G. Shi, A. Liu, “Perceptual Quality Metric with Internal Generative Mechanism”, IEEE Trans. on Image Processing, VOL. 22, NO. 1, pp. 43- 54, 2013 (download) (Source Codes)
Our eyes distinguishes orderly and disorderly visual signals, so does our quality metric.

M. Narwaria, W. Lin and A. Liu, “Low-Complexity VQA Using Temporal Quality Variations”, IEEE Trans. on Multimedia, Vol. 14(3), pp. 525-535, 2012 (download(Source Codes)
Quality changes over time matter more

G. Zhai , X. Wu, X. Yang, W. Lin, W. Zhang, “A Psychovisual Quality Metric in Free Energy Principle”, IEEE Transactions on Image Processing, Vol. 21(1), pp. 41 – 52, 2012, (download)
A fresh angle to look at picture quality prediction issues

W. Lin, C.-C. Jay Kuo, “Perceptual Visual Quality Metrics: A Survey”, J. of Visual Communication and Image Representation, Vol 22(4), pp. 297-312, May 2011 (download)
A comprehensive review of visual quality evaluation: our views
The Most Cited Journal of Visual Communication and Image Representation Article, 2011-2016
L. Ma, W. Lin, C. Deng, and K. N Ngan, “Image Retargeting Quality Assessment: A Study of Subjective Scores and Objective Metrics”, IEEE Journal of Selected Topics in Signal Processing, Vol. 6(6), pp. 626 - 639, 2012 (download) (database)

M. Narwaria, W. Lin, E. Cetin, “Scalable Image Quality Assessment with 2D Mel-cepstrum and Machine Learning Approach”, Pattern Recognition, vol. 45, no. 1, pp. 299-313, 2011,( download)
Attempt for machine-learning in image quality assessment

M. Narwaria, and W. Lin, "Objective Image Quality Assessment Based on Support Vector Regression", IEEE Transactions on Neural Networks, Vol.21(3), pp.515-519, March 2010 (download).
This was among the first attempts to introduce and demonstrate machine learning as a new way of IQA. It avoids the unrealistic assumptions otherwise in the existing feature pooling frameworks.

G. Zhai, J. Cai,  W. Lin, X. Yang,  W. Zhang, M. Etoh, “Cross-dimensional Perceptual Quality Assessment for Low Bitrate Videos”,  IEEE TRANSACTIONS ON MULTIMEDIA, vol.10 (7), pp. 1316-1324, Nov 2008.( download)
The work demonstrates that perceptual visual quality evaluation is a multidimensional problem (content, codec, image size, framerate, bitrate, and so on).

W. Lin, Gauging Image and Video Quality in Industrial Applications, in Advances of Computational Intelligence in Industrial Systems, eds. Y. Liu, et al, Springer-Verlag, Heidelberg, 2008, (download)
Although technology development of visual quality evaluation is still in its infancy, some deployment has been made in industries.

F. Pan, X. Lin, S. Rahardja, E. Ong and W. Lin, “Using edge direction information for measuring blocking artifacts of images”, Multidimensional Systems and Signal Processing, Vol 18 (4), pp. 297 – 308, December 2007, (download)

W. Lin, L. Dong, P. Xue, “Visual Distortion Gauge Based on Discrimination of Noticeable Contrast Changes”, IEEE Trans. Circuits and Systems for Video Technology, vol.15(7), pp. 900- 909, July, 2005, (download)
For long, an unreasonable assumption (and restriction) had always been made in IQA that test images are all distorted.

Firstly, this research enables evaluation for enhanced images/videos, thus removing the said restriction.

Secondly, the proposed measure gracefully reduces to traditional mean-absolute-error (MAE), wherever JND approaches constant and contrast-changes are less differentiated; it allows easier user acceptance and comparison/benchmarking with traditional measures (like MAE, MSE or PSNR).


E. Ong, X. Yang, W. Lin, Z. Lu, S. Yao, X. Lin, S. Rahardja and C. Boon, “Perceptual Quality and Objective Quality Measurements of Compressed Videos”, Journal of Visual Communication and Image Representation, vol.17(4), pp.717-737, August 2006, (download)

E. Ong, W. Lin, Z. Lu, S. Yao and M. Etoh, “Visual Distortion Assessment with Emphasis on Spatially Transitional Regions”, IEEE Trans. Circuits and Systems for Video Technology, Vol. 14(4), PP.559 – 566, April 2004, (download)

F. Pan, X. Lin. S. Rahardja. W. Lin, E. Ong, S. Yao, Z. Lu and X. Yang, “A locally-adaptive algorithm for measuring blocking artifacts in images and videos”, Signal Processing: Image Communication, Vol 19(6), pp. 499-506, 2004, (download)

Authored book:
L. Xu, W. Lin, and C.-C. Jay Kuo, Visual Quality Assessment by Machine Learning, Springer, 2015 (in writing).
Edited book:
C. Deng, L. Ma, W. Lin, and K. N. Ngan (eds.), Visual Signal Quality Assessment – Quality of Experience (QoE), Springer, 2014.
Evalaution of other signals
G. Ghinea, C. Timmerer, W. Lin, S. Gulliver, “Mulsemedia: State-of-the- Art, Perspectives and Challenges”, ACM Transactions on Multimedia Computing Communications and Applications, Vol. 11(1s), Article 17, 2014 (download)
Humans have 5 senses; what can we expect in a machine?
M. Narwaria, W. Lin, I. McLoughlin, S. Emmanue, L. T. Chia, “Nonintrusive Quality Assessment of Noise Suppressed Speech with Mel-Filtered Energies and Support Vector Regression”, IEEE Trans. on Audio, Speech and Language Processing, Vol. 20(4), pp. 1217 - 1232, 2012 (download)
New attempt for speech quality evaluation: the machine-learning approach

Editorials:
W. Lin, T. Ebrahimi, P. C. Loizou, S. Möller, A. R. Reibman, “Introduction to the Special Issue on New Subjective and Objective Methodologies for Audio and Visual Signal Processing”, Editorial, IEEE Journal of Selected Topics in Signal Processing, Vol. 6(6), pp. 614-615, 2012 (download)
W. Zeng and W. Lin, “QoE Modeling and Applications for Multimedia Systems” ZTE Communications, Vol. 11(1), 2013 (download)
T. Dagiuklas, W. Lin and A. Ksentini, “QoE Aware Optimization in Mobile Networks”, IEEE COMSOC MMTC E-Letter, Vol. 8, No. 2, March 2013.
T. Daguiklas, L. Atzori, P. Chatzimisios, C. Chen, W. Lin, “Special issue on QoE in 2D/3D video systems”, Journal of Visual Communication and Image Representation, VOL. 25, NO. 3, pp. 523-534, 2014.
G. Ghinea, C. Timmerer, W. Lin, S. Gulliver, “Special issue on Multiple Sensorial (MulSeMedia) Multi-modal Media: Advances and Applications”, ACM Transactions on Multimedia Computing, Communications and Applications (ACM TOMCCAP), Vol. 11(1s), 2014 (download).
Just-Noticeable Difference (JND) Formulation
To be noticeable, or not to be noticeable: that is the question.

J. Wu, L. Li, W. Dong, G. Shi, W. Lin, C.-C. Jay Kuo, “Enhanced Just Noticeable Difference Model for Images with Pattern Complexity”, IEEE Transactions on Image Processing, 26(6): 2682 - 2693, 2017
J. Wu, G. Shi, W. Lin, A. Liu, F. Qi, “Just Noticeable Difference Estimation For Images with Free-Energy Principle”, IEEE Trans. Multimedia, Vol. 15, No. 7, pp. 1705-1710, 2013, (download)
JND modeling with theoretical support from neuroscience
J. Wu, W. Lin, G. Shi, X. Wang, F. Li, “Pattern Masking Estimation in Image with Structural Uncertainty”, IEEE Transaction on Image Processing, VOL. 22, NO. 12, pp. 4892 - 4904, 2013 (download)
A extended JND modelor from the one above:  with the extraction of disorderly (unpredicted) portion of images.
A. Liu, W. Lin, M. Paul, C. Deng, and F. Zhang, “Just Noticeable Difference for Images with Decomposition Model for Separating Edge and Textured Regions”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 20(11), pp. 1648-1652, 2010 (download) (Source Codes)
Differentiating different visual contents: the key for further JND model improvement
G. Zhai, W. Zhang, X. Yang, W. Lin, Y. Xu, “No-reference Noticeable Blockiness Estimation in Images”, Signal Processing: Image Communication, vol.23 (6), pp. 417-432, July 2008, (download)
X. Zhang, W. Lin and P. Xue, “Just-Noticeable Difference Estimation With Pixels in Images”, Journal of Visual Communication and Image Representation, Vol 19(1), pp 30-41, 2008, (download)
Since JND (just noticeable difference) is a characteristic of images, it should be able to convert from one domain to another. This work demonstartes the conversion from transform domain to pixel one.
 
Y. Jia, W. Lin and A. A. Kassim, “Estimating Just-Noticeable Distortion for Video”, IEEE Trans. Circuits and Systems for Video Technology, vol.16(7), pp. 820- 829, July, 2006, (download(Source Codes)
Formulation of spatiotemporal Contrast Sensitivity Function (CSF) with considearion of eye movement
W. Lin, Computational Models for Just-noticeable Difference, Chapter 9 in Digital Video Image Quality and Perceptual Coding, eds. H. R. Wu and K. R. Rao, CRC Press, 2006, (download)
A general introduction and formulation on visual JND models-- a JND is the minimum amount of change for the difference to be detectable by humans, say 75% of the time. Next questions: what is the implication for 2JND, 3JND, ... ? How to derive them? JND for hearing, touching, smelling or even tasting?  
X. Zhang, W. Lin,  P. Xue, “Improved Estimation for Just-noticeable Visual Distortion”, Signal Processing, Vol. 85(4), pp.795-808, April 2005, (download(Source Codes)
To model the basic image JND in transform domain, with a new formula for i) realistic luminance adaptation according to psychophysical findings in digitized (instead of analog) images, and ii) efficient block classification toward contrast-masking. 
X. Yang, W. Lin, Z. Lu, E. Ong and S. Yao, “Just Noticeable Distortion Model and Its Applications in Video Coding”, Signal Processing: Image Communication, Vol. 20(7), pp. 662-680, August 2005, (download)
The novel and perceptually-plausible nonlinear additivity formula for JND proposed in spatial domain as generalization of all prior work, and demonstrated with its impact for video coding by “killing 3 birds with one stone”: higher perceived-quality, higher PSNR and lower computational-complexity. 
Visual Attention (VA) Modeling
Since William James' time,   the effort to understand and model human attention continues.

T. Xi, W. Zhao, H. Wang, W. Lin, “Salient object detection with spatiotemporal background priors for video”, IEEE Transactions on Image Processing,  26(7): 3425-3436, 2017
K. Gu, G. Zhai, W. Lin, X. Yang, W. Zhang, “Visual Saliency Detection With Free Energy Theory”, IEEE SIGNAL PROCESSING LETTERS, 2015, accepted.
Y. Fang, W. Lin, Z. Fang, Z. Chen, C-W Lin, C. Deng, “Visual Acuity Inspired Saliency Detection by Using Sparse Features”, Information Sciences, accepted
F. Shao, W. Lin, W. Lin, G. Jiang, M. Yu, R. Fu, “Stereoscopic visual attention guided seam carving for stereoscopic image retargeting”, IEEE/OSA Journal of Display Technology, accepted.
Y. Fang, J. Wang, M. Narwaria, P. Le Callet, W. Lin, “Saliency Detection for Stereoscopic Images”, IEEE Transaction on Image Processing,VOL. 23, NO. 6, pp. 2625 - 2636, 2014  (download) (Source Codes)
Y. Fang, W. Lin, Z. Chen, C-M Tsai, C-W Lin, “A Video Saliency Detection Model in Compressed Domain”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 1, pp. 27-38, 2014  (download)
Initial work for compressed-domain VA modeling for video.
H. Tian, Y. Fang, Y. Zhao, W. Lin, R. Ni, Z. Zhu, “Salient Region Detection by Fusing Bottom-up and Top-down Features Extracted from a Single Image”, IEEE Transaction on Image Processing, VOL. 23, NO. 10, pp. 4389-4398, 2014 (download(Source Codes)
X. Bai, Y. Fang, W. Lin, L. Wang, B. Ju, “Saliency-Based Defect Detection in Industrial Images by Using Phase Spectrum”, IEEE Transactions on Industrial Informatics, VOL. 24, NO. 8, pp. 2135-2145, 2014 (download)
An example for possible industrial use of a VA model.
Y. Fang, Z. Wang, W. Lin, Z. Fang, “Video Saliency Incorporating Spatiotemporal Cues and Uncertainty Weighting”, IEEE Transaction on Image Processing, VOL. 23, NO. 9, pp. 3910-3921, 2014 (download(Source Codes)
L. Dong, W. Lin, Y.  Fang, S. Wu, H. S. Seah, “Saliency detection in computer rendered images based on object-level contrast”, Journal of Visual Communication and Image Representation, VOL. 25, NO. 3, pp. 525-533, 2014 (download)
In comparison with the cases of natural images and video, perceptual computer graphics and animation are relatively less investigated so far, in spite of their importance.
Y.K.A. Low, X. Zou, Y. Fang, J.L. Wang, W. Lin, F.Y.C Boey, K.W. Ng, “β-Phase poly(vinylidene fluoride) films encouraged more homogeneous cell distribution and more significant deposition offibronectin towards the cell–material interface compared to α-phase poly(vinylidene fluoride) films”, Materials Science and Engineering C, Vol. 34, pp. 345-353, 2014 (download)
Visual saliency detection used in material engineering.
Y. Fang, Z. Chen, W. Lin, and C-W Lin, "Saliency Detection in the compressed domain for adaptive image retargeting", IEEE Trans. Image Processing, Vol. 21 (9), pp. 3888 - 3901, 2012 (download) (Codes)
The first work for compressed-domain VA modeling, which is significant since all images are stored and transmitted in compressed format; this enables visual saliency being directly determined without the need of decoding, for benefits in cost and power saving. The application of the model to image retargeting is also explored. 

Y. Fang, W. Lin, B-S Lee, C. T. Lau, Z. Chen, C-W Lin, “Bottom-up Saliency Detection Model Based on Human Visual Sensitivity and Amplitude Spectrum”, IEEE Transactions on Multimedia, Vol. 14(1), pp. 187 - 198, 2012, (download(Source Codes)
A visual attention model that considers human visual sensitivity variations due to foveation (i.e., the human visual sensitivity drops fast away from the attention center)

N. Imamoglu, W. Lin, and Y Fang, “A saliency detection model using low-level features based on wavelet transform”, IEEE Trans. Multimedia, Vol. 15(1), pp. 96 - 105, 2013 (download(Source Codes)

Z. Lu, W. Lin, X. Yang, E. Ong and S. Yao, “Modeling Visual Attention's Modulatory Aftereffects on Visual Sensitivity and Quality Evaluation”, IEEE Trans. Image Processing, Vol.14(11), pp.1928 – 1942, Nov. 2005, (download)
Formulating VA’s modulatory effect on overall visual sensitivity in video, with a well-grounded basis to integrate bottom-up and top-down stimuli, according to relevant physiological and psychological knowledge.
Authored book:
L. M. Zhang, W. Lin, Modeling Selective Visual Attention: Techniques and Applications, John Wiley & Sons, 2013.

book
Th
is book fills the gap of the basic VA theory and real-world applications (e.g., image retrieval, compression, retargeting, recognition, compressive sensing, IQA, and robotics), and provides a thorough and systematic cover, backed by the intensive research by the authors and the teams they led during the past 15 years.
Perceptual Visual Coding & Processing
We are making every bit and operation count, and count for users.
 

K. Gu, G. Zhai, W. Lin, M. Liu, “The Analysis of Image Contrast: From Quality Assessment to Automatic Enhancement”, IEEE Transactions on Cybernetics, accepted
V. Jakhetiya, W. Lin, S. P. Jaiswal, S. C. Guntuku, O. C. Au, “Maximum a Posterior and Perceptually Motivated Reconstruction Algorithm: A Generic Framework”, IEEE Trans. Multimedia,  19(1): 93 - 106, 2017
F. Shao, W. Lin, G. Jiang, M. Yu, “Low-complexity depth coding by depth sensitivity aware rate-distortion optimization”, IEEE Trans. on Broadcasting, accepted
S. Wang, K. Gu, S. Ma, W. Lin, W. Gao, “Guided Image Contrast Enhancement Based on Retrieved Images in Cloud”, IEEE Trans. Multimedia, accepted, 2015
F. Shao, W. Lin, G. Jiang, M. Yu, Q. Dai, “Depth Map Coding for View Synthesis Based on Distortion Analyses”, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, VOL. 4, NO. 1, pp. 106-117, 2014 (download)
H. Bai, W. Lin, M. Zhang, A. Wang, Y. Zhao, “Multiple Description Video Coding Based on Human Visual System Characteristics”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 8, pp. 1390-1394, 2014 (download)
Use of a JND model in multiple-description video coding.
L. Dong, Y. Fang, W. Lin, C. Deng, C. Zhu, H. S. Seah, “Exploiting entropy masking in perceptual graphic rendering”, Signal Processing: Image Communication, accepted, 2015.
Unlike in the cases of natural images and video, we do not have the original visual signals as the reference in computer graphics, so perception is the only criterion for processing.
H. R. Wu, A. Reibman, W. Lin, F. Pereira, S. S. Hemami, “Perceptual Visual Signal Compression and Transmission”, PROCEEDINGS OF THE IEEE, VOL. 101, NO. 9, pp. 2025 - 2043, 2013 (download)
A survey for perceptual image/video coding and transmission.
F. Shao, G. Jiang, W. Lin, Y. Mei, Y. H. Dai, “Joint  Bit Allocation and Rate Control for Coding Multi-view Video plus Depth based 3D Video”, IEEE Trans. Multimedia, Vol. 15(8), pp. 1843 - 1854, 2013 (download)
Y. Gao, X. Xiu, J. Liang, W. Lin, “Fast Synthesized and Predicted Just Noticeable Distortion Maps for Perceptual Multiview Video Coding”, Journal of Visual Communications and Image Representation, VOL. 24, NO. 6, pp. 700-707, 2013 (download).
Use of a JND model in multiview video coding.
M. Paul, W. Lin, “Computer vision aided video coding”, in Advanced Video Communications Over Wireless Networks, C. Zhu and Y Li (eds.), CRC Press, 2012.
F. Zhang, W. Liu, W. Lin, K. N. Ngan, “Spread Spectrum Image Watermarking Based on Perceptual Quality Metric”, IEEE Transaction on Image Processing, VOL. 20, NO. 11, pp. 3207 – 3218, Nov 2011, (download) (Source Codes)
S. Wu, S. Xie, and W. Lin, “Blind Measurement of Image Blur for Vision-based Applications”, Multimedia Analysis, Processing and Communications, eds. W. Lin, et al., pp.185-215, Springer, 2011( download) (Source Codes)
Visual quality evaluation can be used as pre-processing for computer vision based applications.
S. Wu, W Lin,and S. Xie,  Z. Lu, E. Ong, S. Yao, “Blind Blur Assessment for Vision-based Applications”, Journal of Visual Communication and Image Representation, Vol 20(4), pp 231-241, 2009, (download) (Source Codes)
W. Lin, Y. Gai and A. A. Kassim, “A Study on Perceptual Impact of Edge Sharpness in Images”, IEE Proc. Vision, Image & Signal Processing, vol. 153(2), pp. 215-223, April 2006, (download)
To which extent can we enhance image edge?
X. Yang, W. Lin, Z. Lu, E. Ong, S. Yao, “Motion-compensated Residue Pre-processing in Video Coding Based on Just-noticeable-distortion Profile”, IEEE Trans. Circuits and Systems for Video Technology, vol.15(6), pp.742-750, June, 2005, (download(Source Codes)
Much work has been done to optimize coders, and here we propose to optimize signals for better compressibility.
X. Yang, W. Lin, Z. Lu, X. Lin, S. Rahardja, E. Ong, S. Yao, “Rate Control for videophone using perceptual sensitivity cues”, IEEE Trans. Circuits and Systems for Video Technology, vol 15(4), pp.496-507, April, 2005 (download)
Other Image/Video Processing Topics
X. Zhang, W. Lin, S. Ma, S. Wang, W. Gao, “Low-Rank based Nonlocal Adaptive Loop Filter for High Efficiency Video Compression”, IEEE Trans. Circuits and Systems for Video Technology, accepted.
W. Cheng, W. Lin, X. Zhang, M. Goesele, M-T Sun, “A Data-driven Point Cloud Simplification Framework for City-scale Image-based Localization”, IEEE Transactions on Image Processing,  26(1): 262-275, 2017.
H. Yang, S. Wu, C. Deng, W. Lin, “Scale and Orientation Invariant Text Segmentation for Born-Digital Compound Images”, IEEE Transactions on Cybernetics, VOL. 45, NO. 3, pp. 533 - 547, 2015 (download) 
Screen images are increasingly important, so is the processing for them.
Y. Yuan, Y. Fang, W. Lin, “Visual Object Tracking by Structure Complexity Coefficients”, IEEE Trans. Multimedia, 17(8):1125 - 1136, 2015.
Target appearance change is a major challenge in object tracking. In this work, structural and smooth regions are differentiated in similarity evaluation of target appearance, due to their inherent difference in appearance stability.
X. Zhang, R. Xiong, W. Lin, S. Ma, J. Liu, W. Gao, "Video Compression Artifact Reduction via Spatio-Temporal Multi-Hypothesis Prediction", IEEE Transactions on Image Processing, accepted.
S. Wang,  Z. Wang, S. E. M. Foo, N. S. Tan,  Y. Yuan, W. Lin, Z. Zhang, K. W. Ng, “Culturing Fibroblasts in 3D Human Hair Keratin Hydrogels”, ACS Applied Materials & Interfaces, accepted
Y. Yuan, S. Emmanuel, Y. Fang, W. Lin, “Visual Object Tracking based on Backward Model Validation”, IEEE Transactions on Circuits and Systems for Video Technology, VOL. 24, NO. 11, pp. 1898 - 1910, 2014 (download)
Occusion is another major challenge in object tracking: backward (i.e., using future frames) model validation helpful in differentiating occlusion and large appearance change.
L. Zhang, L. Wang, W. Lin, S. Yan, “Geometric Optimum Experimental Design for Collaborative Image Retrieval”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 2, pp. 346 - 359, 2014 (download) 
J. Wu, W. Lin, G. Shi, J. Xiao, “Correlation based Universal Image/Video Coding Loss Recovery”, Journal of Visual Communication and Image Representation,VOL. 25, NO. 7, pp. 1507–1515, 2014, (download) 
Using the correlation among coded pixels to reduce coding loss.
M. Paul, W. Lin, C. T. Lau, B-S Lee, “A Long Term Reference Frame for Hierarchical B-Picture based Video Coding”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 24, NO. 10, pp. 1729-1742, 2014 (download)
Since a natural video frame cannot be an ideal reference frame for coding in general,  we generate a man-made one to serve the purpose.
W Liu and W Lin, “Gaussian Noise Level Estimation in SVD Domain for Images”, IEEE Transaction on Image Processing, VOL. 22, NO. 3, pp. 872 - 883, 2013, (download(Source Codes)
The SVD provides a good a basis for noise level estimation in images: the signal contribution decreases rapidly in singular values, while the noise contribution remain throughout singular values -- the "tail" of singular values tells about the noise! 
C-M Tsai, L-W Kang, C-W Lin, W. Lin, “Scene-Based Movie Summarization via Role-Community Networks”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, Vol. 23(11), pp. 1927 - 1940, 2013, (download)
M. Paul, W. Lin, C. T. Lau, B-S Lee, “Video Coding with Dynamic Background”, EURASIP Journal on Advances in Signal Processing, 11, 2013 (view)
Z. Gu, W. Lin, B-S Lee, C. T. Lau, M-T Sun, “Mode Dependent Templates and Scan Order for H.264/AVC Based Intra Lossless Coding”, IEEE Transaction on Image Processing, Vol. 21(9), pp. 4106 - 4116, 2012 (download)
How to engineer more zeros in bitstream?
 
L. Zhang, L. Wang, W. Lin, “Conjunctive Patches Subspace Learning with Side Information for Collaborative Image Retrieval”, IEEE Transactions on Image Processing, Vol. 21(8), pp. 3707 - 3720, 2012, (download)
Z. Gu, W. Lin, B.-S. Lee, C. T. Lau, “Low Complexity Video Coding Based on Two-dimensional Singular Value Decomposition (2D-SVD)”, IEEE Trans. on Image Processing, Vol. 21(2), pp. 674 - 687, 2012 (download)
2D-SVD as the transform for coding

C. Deng, W. Lin, B-S  Lee, C. T.  Lau, “Robust Image Coding Based upon Compressive Sensing”, IEEE Transactions on Multimedia, Vol. 14(2), pp. 278 - 290, 2012 (download)

C. Deng, W. Lin, and J. Cai, “Content-based Image Compression for Arbitrary-resolution Display Devices”, IEEE Trans. Multimedia, Vol. 14(4), pp. 1127 - 1139, 2012 (download)
We make a single bit stream equipped with decoding possibility of arbitrary image sizes.

L. Zhang, L. Wang, W. Lin, “Semi-supervised biased maximum margin analysis for interactive image retrieval”, IEEE Transactions on Image Processing, Vol. 21(4), pp. 2294 - 2308, 2012.
Z. Gu, W. Lin, B-S Lee, C. T. Lau, “Rotated Orthogonal Transform (ROT) for Motion-Compensation Residual Coding” IEEE Transaction on Image Processing, VOL. 21, NO. 12, pp. 4770 – 4781, DECEMBER 2012 (download)
If we worry about the following two aspects of transforms used in video coding:
i) a traditional (and fixed) transform (like DCT or Wavelet) cannot be optimal for all video, and
ii) a adaptive trasform (like SVD, etc.) needs to specify the transform itself (this causes bits),
then this work is a solution: start with DCT (
guaranteeing a better result than DCT), and rotate DCT basis fuctions adaptively for visual content (the rotation can be specified efficiently).

L. Zhang, L. Wang, W. Lin, “Generalized Biased Discriminant Analysis for Content-Based Image Retrieval”, IEEE Transactions on Systems, Man, and Cybernetics--Part C, Vol. 42(1), pp. 282 - 290, 2012, (download)
Our way to tackle the problems of the positive within-class scatter and the Gaussian distribution assumption for positive samples, in Discriminant Analysis (BDA)
M. Liu, HS Seah, C. Zhu, W. Lin, F. Tian, “Reducing Location Map In Prediction-Based Difference Expansion For Reversible Image Data Embedding Signal Processing”, Signal Processing, Vol. 92, Issue 3, pp. 819-828, March 2012, (download)
A. Liu, W. Lin, M. Paul, F. Zhang, C. Deng, “Optimal Compression Plane for Efficient Video Coding”, IEEE Transaction on Image Processing, 20(10), pp.2788-2799, 2011, (download) (Source Codes)
"Out of the box" thinking: who says that a frame has to be in the XY plane always?

M. Paul, W. Lin, C. T. Lau, B. –S. Lee, “Explore and Model Better I-frames for Video Coding”, IEEE Trans. Circuits and Systems for Video Technology,Vol. 21(9), pp. 1242 – 1254, 2011, (download)(Source Codes)
A "golden frame" we make for motion estimation and as an I frame, since a natural frame cannot do the two jobs well.

M. Paul, W. Lin, C. T. Lau, B. –S. Lee, “Direct Inter-Mode Selection for H.264 Video Coding using Phase Correlation,” IEEE Transaction on Image Processing, 20(2), pp. 461-473, Feb. 2011, (download)

C. Deng, W. Lin, B-S  Lee, C. T.  Lau, M-T  Sun, "Performance Analysis, Parameter Selection and Extensions to H.264/AVC FRExt for Super-High Definition Video Coding”, Journal of Visual Communication and Image Representation, Vol.22(8), pp. 749-759, 2011, (download)
Does H.264 still work well for coding HD video? Yes, if you read this paper:-).

G. Zhai, X. Yang, W. Lin and W. Zhang, “Bayesian Error Concealment with DCT Pyramid for Images”, IEEE Trans. Circuits and Systems for Video Technology, Vol 20(9), pp. 1224-1232, Sept  2010, (download)
Choose the right level to start making a sensible decision...for image restoration

M. Shen, C. Wang, P. Xue and W. Lin, “Performance of Reconstruction-based Super-resolution with Regularization”, Journal of Visual Communications and Image Representation, Vol.21(7), pp.640-650, Oct. 2010(download)
G. Zhai, W. Zhang, X. Yang, W. Lin and Y. Xu, “Unified Deblocking for DCT Compressed ”, J. of Electronic Imaging, Vol. 17(4), 2009, (download)

G. Zhai, W. Lin,  J. Cai , X. Yang  and W. Zhang, “Effective Quadtree based Block-shift Filtering for Deblocking and DeringingJournal of Visual Communication and Image Representation, vol.20(8), pp.595-607, Nov 2009, (download)

P. Li, W. Lin, X.Yang, “Analysis of H.264/AVC and an Associated Rate Control Scheme”, J. of Electronic Imaging, Vol. 17(4), 2009, (download)

G. Zhai, W. Zhang, X. K. Yang, W. Lin, and Y. Xu, “Efficient deblocking with coefficient regularization, shape adaptive filtering and quantization constraint”, IEEE Transactions on Multimedia, vol. 10(8), pp. 735-745, 2008, (download)

S. Wu, W Lin, and S. Xie, “Skin Heat Transfer Model of Facial Thermograms and Its Application in Face Recognition”, Pattern Recognition, Vol. 41(8), pp. 2718-2729, 2008, (download)
Blood perfusion rate reveals more about ourselves, so facilitates better face recognition.   
G. Zhai, W. Zhang, X. Yang, W. Lin and Y. Xu, “Efficient Image Deblocking Based on Postfiltering in Shifted Windows”, IEEE Trans. Circuits and Systems for Video Technology. Vol. 18 (1), pp.122 – 126, Jan. 2008, (download)

C Wang, P. Xue, W. Lin, “Layered Image Resizing in Compression Domain”, Signal Processing: Image Communication, Vol.23 (1) , pp. 58-69, January 2008, (download)

G. Zhai, J. Cai , W. Lin,  X. Yang,  and W. Zhang, “Three Dimensional Scalable Video Adaptation via User-end Perceptual Quality AssessmentIEEE Trans. Broadcasting, Vol. 54(3), pp. 1316-1324, 2008, (download)

E. Ong, W. Lin, “Video Object Segmentation”, in Encyclopedia of Information Communication Technology (ICT) , Ideal Group Inc.,  A. Cartelli and M. Palma (eds.), 2008,(download)
H. Gao, J. Tham, P. Xue and W. Lin, “Complexity Analysis of Morphological Area Openings and Closings with Set Union”, IET Image Processing, Vol. 2, No. 4, pp. 231-238, August 2008, (download)
W. Lin and L. Dong, “Adaptive Down-sampling to Improve Image Compression at Low Bit Rates”, IEEE Trans. Image Processing, Vol.15(9), pp. 2513-2521, Sept. 2006, (download(Source Codes)
When bits become too few to code all pixels, it is better to downsample while encoding and then restore them at the receiving end.

C. Wang, P. Xue, W. Lin, W. Zhang and S. Yu, “Fast Edge-Preserved Postprocessing for Compressed Images”, IEEE Trans. Circuits and Systems for Video Technology, vol.16(9), pp.1142-1147, Sept. 2006, (download)

C. Wang, P. Xue, W. Lin, “Improved Super-resolution Reconstruction from Video”, IEEE Trans. Circuits and Systems for Video Technology, Vol.16(11), pp.1411-1422, Nov. 2006, (download)

E. Ong, W. Lin, B. Tye and M. Etoh, Video Object Segmentation for Content-based Applications, in Advances in Image and Video Segmentation, Chapter VII, ed. Y.J. Zhang, Idea Group, Inc., 2006

H. Gao, W. Lin, P. Xue and W.C. Siu, “Marker-Based Image Segmentation Relying on Disjoint Set Union”, Signal Processing: Image Communication, Vol. 21(2), pp. 100-112, 2006, (download)

P. Li, W. Lin, S. Rahardja, X. Lin, X.K. Yang and Z.G. Li, “Geometrically Determining the Leaky Bucket Parameters for Video Streaming over Constant Bit-Rate Channels”, Signal Processing: Image Communication, Vol. 20(2), pp.193-204, February 2005, (download)
Edited book:
W. Lin, D. Tao, J. Kacprzyk, Z. Li, E. Izquierdo, H. Wang (eds.), Multimedia Analysis, Processing and Communications, Springer, 2011.
W. Lin, D. Xu, A. Ho, J. Wu, Y. He, J. Cai, M. Kankanhalli, M-T Sun (eds.), Advances in Multimedia Information Processing – PCM 2012, Springer, 2012.
Editorials:
D. Xu, W. Lin, A. T. S. Ho, “Advances in multimedia content analysis and signal processing”,  Journal of Signal Processing Systems, 74(1), 1-3, 2014.
Machine Learning & Applications
S. C. Guntuku, J. T. Zhou, S. Roy, W. Lin, I. W. Tsang “Who likes What, and Why? Insights into Personality Modeling based on Image `Likes'”, IEEE Transactions on Affective Computing, accepted
S. I. Niwas, W. Lin, C. K. Kwoh, C.-C. Jay Kuo,   C. C. Sng, M. C. Aquino, Paul T. K. Chew, “Cross-examination for Angle-Closure Glaucoma Feature Detection”, accepted by IEEE Journal of Biomedical and Health Informatics.
S. I. Niwas, W. Lin, X. Bai, C. K. Kwoh, C. C. Sng, M. C. Aquino, P.T.K. Chew, “Reliable Feature Selection Technique for Automated Angle Closure Glaucoma Mechanism Detection”, Journal of Medical Systems, Vol. 39, 2015.
H. K. Y. Choi, W. Lin, S. C. Loon, C. Tan, W. Wong, J. See, Z. Gu, C. K. Kwoh, P. Chew, “Facial Scanning With a Digital Camera: A Novel Way of Screening for Primary Angle Closure”, Journal of Glaucoma, 17 Oct 2013
C. Zhang, W. Bian, D. Tao, W. Lin, “Discretized-Vapnik-Chervonenkis Dimension for Analyzing Complexity of Real Function Classes”, IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, Vol. 23(9), pp.  1461 – 1472, 2012 (download)
A. Wirawan, C. K. Kwoh, P. T. K. Chew, M. C. D. Aquino, S. C. Loon, J. See, C. Zheng, W. Lin, “Feature Selection for Computer-Aided Angle Closure Glaucoma Mechanism Detection”, Journal of Medical Imaging and Health Informatics, Vol. 2, 2012 (download)
Patents
W. Lin, B. Tye, E.P. Ong, “Dynamic Load-balancing between Two Processing Means for Real-time Video Encoding”, US patent 6,748,019 B1 (2004); Singapore patent 9902417-6 (1999).
Z. Lu, W. Lin, S. Yao,  E. Ong, “Method for generating a quality oriented significance map for assessing the quality of an image or video”, US Patent number 7,590,287 B2 (2009), PCT Patent number WO2004/043054 A2 (2004).
Z. Lu, W. Lin, Z. Li, K. P. Lim, X. Lin, S. Rahardja, E. Ong, S. Yao, “Method for encoding a picture, computer program product and encoder”, US Patent Application No. 11/910,929, 2007.
E. Ong, W. Lin, Z. Lu, S. Yao, X. Yang, “Image and Video Quality Measurements”, Singapore Patent Application No. 200307620-5 (2003); PCT Patent number WO 2005/060272 A1 (2005).
E Ong, X Yang, W Lin, Z LU, S Yao, “METHOD AND SYSTEM FOR VIDEO QUALITY MEASUREMENTS”, European Patent 1,692,876, 2006; also WO2005055618 (A1) and US2007257988 (A1).
Z. Gu, W. Lin, B.-S. Lee, C. T. Lau, “Mode Dependent Templates for H.264/AVC Lossless Intra Coding”, US Provisional Patent Application, 61/553,372, October 2011.
A. Liu, W. Lin, H. Chen, P. Zhang, “Image Retargeting Quality Assessment”, filed for USA patent application no: 13/713,110, 2013.

More Publications