Module 1 – Introduction Examples of poor technical image quality: Aliasing (moiré or glitch) is seen in approximately flat. Adaptive histogram equalization breaks your image into small
Digital Images: The values of digital images are all discrete and integers. The values sampled systems. When the input frequency is greater than half the sample blocks, and then does histogram equalization in these small blocks.
can be considered as a large (numpy) array of discrete dots. Each dot has a frequency (or: when the signal is sampled at less than twice the highest frequency),
brightness associated with it. These dots are called picture elements (= pixels). the samples points do not adequately represent the input signal. Inputs at these
higher frequencies are observed at a lower, aliased frequency.
Neighborhood: A neighborhood can be characterized by its shape (e.g., 3X5 Image Quality Affects Deep Neural Networks: - Blurring – Noise - Low contrast -
neighborhood). Usually, the neighborhood is an odd number. If the -In medical imaging only the ultrasound picture is not coming from an JPEG distortion. If they’re higher, worse the neural networks perform. So, before
e)Contrast Stretching: can be used to distribute the intensity range. Note that, in
neighborhood is an even number, interpolation is involved. electromagnetic spectrum. you run your classifiers, you can apply image enhancement to improve your
contrast to gamma transformations, for contrast stretching (rescale intensity), the
Saturation and Noise: Saturation is the highest level of intensity that you can reach. downstream deep neural networks training. values do not have to be floating.
If there is a lot of saturation, you cannot see much detail because it is too bright. Image Noise: Noise is any degradation in the image signal, caused by external
2-Spatial Filtering: Masks: You overlay a mask (in this case a 3x5 mask) over your
You can use contrast enhancement to improve your image. You can also have disturbance. Note that image noise can occur when the image is compressed as well,
pixel to apply an operation. If you have an averaging mask, and e is your pixel, the
images that have noise. Noise exists when there is not enough light. not only when the image is acquired.
pixels surrounding e (a, b, c, …, i) are used for the operation. Spatial filtering is an
Intensity resolution: is the smallest change of intensity in your image and it is Salt & Pepper Noise: The intensity of some of the pixels has been randomly
example of a convolution operation. You also apply convolutions in neural
referred to by the number of bits. The most common is 8-bit (255 grayscale levels). changed to white and black (rather than the original intensity). networks. How do convolutions work? Convolutions require 3 steps: 1. Position
Gaussian Noise: Also called white noise. The intensity of some of the pixels has
the mask over the current pixel; 2. Form all products or filter elements with the
fluctuated.
corresponding elements of the neighbourhood; 3. Add up all the products. These 3
Types of Images: Speckle Noise: Typically occurs when the image is acquired by radar, such as
steps are usually repeated for every pixel in the image. This is called linear filtering:
-Binary: Each pixel is either black (0) or white (1). You only need one bit to military operations. Each pixel is multiplied by some random value.
process is applied one by one in the image and summed up. (Convolution vs.
represent the pixel. Image Blur: Out-of-focus and Motion blur.
Correlation: only difference is conv. Rotated the kernel(mask) by 180 degrees.
-Grayscale: Each pixel is a shade of gray. Normally the values range from 0 (black) Aliasing: You acquire the signal at an interval that is slightly less than the original
Spatial resolution is the density of pixels in the image. The greater the spatial 2a) Smoothing Filters: Linear: In simple average every pixel has same weight like
to 255 (white). Each pixel can be represented by eight bits, or exactly one byte. signal. Aliasing in Images: The image is resized by taking every
resolution, the more pixels are used to display the image, and the image tends be 1/9. However in weighted average centre or pixels near centre has more weight.
-Color: Multi-channel images are a stack of multiple matrices; representing the other pixel (1:2 resize). If a repetitive pattern of high frequency is sampled
Gaussian filter: Based on standard deviation smoothened the image. Non-linear
multiple channel values for each pixel. E.g., RGB color is described by the amount clearer. at a low resolution, more patterns results.
Interpolation: You can use interpolation to reduce or increase the image size. filter: you put a mask of over an image,and take the median value. Used to correct
of red, green and blue in it. Image Enhancement: 1. Spatial Domain (Image Domain): based on direct
- Halve the size of your image: take out every other row and every other column salt and pepper noise because of the nonlinear character of this smoothing filter.
manipulation of pixels in an image (Point Operations and Spatial Filtering
- Double the size of the image: all pixels are repeated to produce an image with the same /Convolutions. 2. Frequency Domain: filtering based on modifying the Fourier 2b) Sharpening filters: You can sharpen an image by taking the derivative. Laplacian
Color Models:
size as the original, but with half the resolution in each direction. filter: second-order derivative filter that is used to enhance edges in an image.
-Red-Green-Blue (RGB) transform of an image.
Color Look Up Tables and Colormaps: Operations that change the display without Unlike first-order methods like the Sobel or Prewitt filters which detect edges by
-Cyan-Magenta-Yellow-Black (CMYK):- Additive colors can be mixed to produce the Image Enhancement in the Spatial Domain: 1-Point operations:
colors: Cyan, Yellow, and White.- Subtractive colors can be mixed to produce the changing the image are: LUT (colormap) and contrast stretching. Operations that a)Intensity Thresholding: Intensity thresholding converts a grayscale image to binary finding the intensity gradients in an image, the Laplacian filter detects regions of
change the image can be reversible and non-reversible (majority). rapid intensity change more directly, which are typically indicative of edges. It is
colors: Red, Green, Blue and Black. image (pixel by pixel): 1. Choosing a gray level T in the original image;2. A pixel
C = Cyan: green + blue = white – red Unsigned vs signed integer uint8 and int8 are very similar, but there are important becomes white if its gray level is > T; 3. A pixel becomes black if its gray level is <= extremely sensitive to noise. 2c) Edge Detection Filters: Prewitt and Sobel:
differences. int8 has range of –127 to 127. uint8 has a range of 0 to 255 (u = 2d) Image Pyramids: Gaussian Pyramid: Reduces the image size and blurs it,
M = Magenta: red + blue = white – green T.
unsigned). which is used for image downscaling and multi-resolution analysis.
Y = Yellow: red + green = white – blue
Arithmetic Operations:These operations applying a simple function y=f(x) to each Laplacian Pyramid: Is a bandpass version of the image pyramid, used for
K = Black
gray value in the image (e.g., add, subtract, etc.). Simple functions include:- adding reconstructing an image from its lower resolution. In-class Quiz Questions-
or subtract a constant value to each pixel - adding or subtract an image from Gaussian noise is: white noise with a mean of zero and a standard deviation of
another image of the same size (width and height and channels) - multiplying each sigma.-Aliasing occurs when a signal is sampled: at slightly less than the time
pixel by a constant: y = C·x - complement: for a grayscale image, you reserve the b)Otsu’s method: - Assumes that the image contains two class of pixels: backgroundinterval of the original signal. -Which of these are true :a. Contrast stretching
bright and dark values To be able to apply arithmetic operations, images have to be (Group A) and foreground (Group B). The foreground is typically what you are stretches the range of intensity values it contains to make full use of possible values
the same size. This is because of linear algebra: columns and rows need to have the interested in (e.g., the rice grains in the image of the practical lecture); - Minimizes b. Contrast stretching is sensitive to outliers (e.g., a small number of pixels with high
same size. the spread (intra-class variance); the distributions (mean, variance, SD) of the and low intensities due to salt and pepper noise)c. Histogram equalization is not
-HSV (Hue, Saturation,Value) - Hue: the “true color” attribute (red, green, blue, Grayscale conversions in PIL and scikit-image : intensities of the two groups; sensitive to outliers. In a bright image, the components of histogram are
orange, yellow, and so on).- Saturation: the amount by which the color has been - In scikit image, the value of each grayscale pixel is calculated as the weighted sum - Maximizes inter-class variance (the difference between the mean of concentrated on which side of the grey scale? Answer: High You should normalize
diluted with white. The whiter the color, the lower the saturation.- Value: the degree of the corresponding red, green and blue pixels as: Group A and the mean of Group B). your image (scale between 0 and 1) for the following intensity transformations: Log
of brightness – a well-lit color has high intensity; a dark color has low intensity. Y = 0.2125 R + 0.7154 G + 0.0721 B c)Adaptive Thresholding: “Local” thresholds calculated with fixed block_size and Gamma transformations. Which of the following filters are used for
-HSL (Hue, Saturation,Luminosity): In the HSV model, These weights are used by CRT phosphors as they better represent human surrounding each pixel (i.e., local neighborhoods). Adaptive thresholding produces smoothing? Averaging and Gaussian Filter. Which of the following filter(s) are used
the value is the degree of brightness. A well-lit color has a high intensity (value); a perception of red, green and blue than equal weights (i.e., 0.3; 0.3; 0.3) better results when there is a large variation in background intensity. Adaptive for sharpening? Laplacian Filter.
dark color has a low intensity (value). The original hue value has a luminosity of - In PIL, a color image is converted to grayscale (mode “L”), using the ITUR 601-2 thresholding is similar to adaptive histogram, in the sense that you break your image Module 3 Image Enhancement in the Frequency Domain
128. When decreasing luminosity (minimum is 0), the color becomes darker (shade). luma transform: into small sections and apply localized thresholding. Fourier Transforms Convolution in an image domain is equivalent to a simple
When increasing luminosity (maximum is 255), the color becomes brighter (tint). L = 0.299 R + 0.587 G + 0.114 B multiplication in the frequency domain. Multiplication in the frequency domain is
Why do we convert images to greyscale? Traditional image analysis algorithms faster especially when the kernel and/or the image is large. Allows a signal to be
especially image enhancements algorithms, tend to work on only one channel such decomposed into a sum of sine waves of different frequencies, phases and
that you can convert to greyscale, or you can apply the three (R, G, B) channels amplitudes. Fourier transform make images sine and cosine waves instead of pixels.
separately. Second picture is harmonics.
Fundamental Steps in IA ◦ Frequencies: amount by which grey values change with distance.
Image Acquisiton-Preprocessing-Morphological Operations-Image ◦ High frequency components - large changes in grey values over small distances;
Segmentation-Object Recognition-Quantification-Visualization (edges and noise)
d)Gamma Transformations: When γ=1, the transformation is linear, and the output
image is the same as the input image. If γ<1, the transformation is sub-linear, and ◦ Low frequency components - by little change in the gray values. (backgrounds,
In-class quiz questions : the output image will appear lighter, which can enhance details in dark regions. skin textures)
1. The maximum intensity level in an 8bit image is 255. Conversely, if γ>1, the transformation is super-linear, and the output image will
2. The red color in a hue image will correspond to 0. appear darker, which can enhance details in bright regions.(Note that you should never
3. Each element (point) of an image matrix (numpy array) is called a pixel. apply gamma transformations to unsigned integer images, because your image intensity should be
4. Black color in a uint8 (unsigned 8bit) image is usually represented by the value 0. between 0 (completely dark) and 1 (completely bright). That is why you should convert your image
5. The maximum intensity level in a 4bit image is 15. to float, and normalize the values between 0 and 1, before applying gamma transformation.)
Previous Exam Question 1: Which channel is the Hue Channel? Justify your
All the bands (e.g., X-ray, radio, visual), are part of the electromagnetic spectrum.
Each band has a certain wave length. The main source of energy is radiation from
electromagnetic waves. They are massless particles moving at the speed of light, and e)Intensity Histograms: The histogram of a digital image indicates how the intensity
they contain energy. Gamma rays have a very short wave-length, and are typically of pixels distributes among the whole pixels (how many pixels are at a certain
used in PET machines and PET deposits for positron emission tomography intensity range). An intensity histogram shows the darkness, brightness and contrast
answer.
machines. Radio waves have very long wave lengths. Images coming from radio
Fig. (b) Image 1 is the hue channel. The red has a value of 0 [2 pts], the value for of an image.
waves belong to astronomy. Between UV and infra-red is the visible spectrum. It is ◦ High pass filters passes over the high frequency components and reduces or
yellow is higher than red but lower than green. The green channel is darker than the
a band with wavelengths around 450 nanometers to 750 nanometers. This is the eliminates low frequency components.
blue channel and the magenta channel is the brightest [2 pts]. These intensity values
band that we are most familiar with. This band consists of many colors from red
correspond to the hue scale. As for the other channels: Fig. (c) Image 2 is the f)Histogram Equalization: The global histogram may not be a good representation ◦ Low pass filters passes over the low frequency components and reduces or
(with a longer wavelength) to blue (with a lower wavelength). This band is easiest to
saturation channel, because the intensities for all the colors are nearly the same. Fig. for local statistics in two parts of the image. We can apply histogram equalization in eliminates high frequency components.
distinguish because of the many identifiable colors.
(d) Image 3 is the blue channel, because the intensities where the pixel contain blue such a way that the number of pixels per intensity range is equal. It transforms the ◦ Band pass filters passes frequencies within a certain range and rejects (attenuates)
are brighter. White and cyan and magenta contain blue. intensities of your pixel in such a way that the histogram of your output image is frequencies outside that range.
Module 2 - Image Enhancement in the Spatial Domain Low Pass (Smoothing) Filters