Summary
Summary of notebooks Data Processing advanced
- Course
- Institution
Summary of notebooks Data Processing advanced
[Show more]
Seller
Follow
sophiedekkers54
Reviews received
Content preview
Summary Data processing advancedWeek 1
Array = ordered and structured collection of elements.
- Arrays are structured around the number of dimensions they contain.
1-dimensional array vector (e.g., stock price)
2-dimensional array matrix (e.g., gray scale image)
N-dimensional array tensor (e.g., RGB image)
How to create an array
My_list = [1, 3, 5, 7, 9]
My_array = numpy.array(my_list)
- My_array outputs [1 3 5 7 9]
You could also create an array without specifying a python list in advance:
My_array = numpy.array([2, 4, 6, 8])
Dtype() specifies the datatype for the elements inside the array.
Creating an array from a file
Numpy.loadtxt() loads a text file.
Numpy.load() loads a binary file.
Numpy.savetxt() saves an array in a file
numpy.savetxt("sample_file.txt", a)
it saves the array (a) to the text file specified
creating arrays based on range
numpy.arange(a, b, s) a is the start point, b the endpoint and s is the step size.
Can have integers or floats as inputs.
So, useful when you want to specify the stepsize.
Numpy.linspace(a, b, i) a is the start point, b the endpoint and I is the number of
items.
Useful when you want the specify the amount of items in the range.
Creating arrays of random values
Numpy.random.random(n) returns n numbers between 0 and 1
Numpy.random.uniform(x, y, z) x is the starting point, y the endpoint and z the
number of items drawn between x and y.
Numpy.random.randint(x, y, z) returns z random integers between x and y.
For a normal distribution you can use:
Numpy.random.normal(x, y, z)
,Shape and size
Examples for array a:
A.size shows the total number of element in the array
A.ndim shows the number of dimensions.
To check if two arrays are the same you could check:
Type() looks if the object types match.
Numpy.allclose() looks if the elements all match.
Id() looks if the location in computer memory is the same for both objects.
Immutable datatypes python:
Boolean
Integer
Float
Complex
String
Byte
Tuple
Mutable datatypes:
List
Set
Dictionary
You can specify the datatype of the array when you create the array with the array() function
using the dtype() argument.
- Note: not every combination is possible.
- Example:
strings = ["12", "3", "24"]
z = numpy.array(strings, dtype='int')
you can use the astype() function to convert an existing numpy array to a different type.
- Sometimes, the data types are converted but the content is slightly changed. For
example, when converting from float to integer, then the numbers are rounded down
(floor). For example, when converting an array with only zeros and ones to the data
type Boolean than the 0 is converted to False and the 1 is converted to True.
Example from something that gives an error:
- x = numpy.array(["true", "false"])
- a = x.astype('bool')
indexing a 1-dimensional array is the same as indexing a python list.
A = numpy.arange(10) [0 1 2 3 4 5 6 7 8 9]
Print(a[0]) 0
Print(a[3]) 3
A function can also return a linear index. Two examples are:
- argmin
- argmax
these return the index of the minimal value and index of the maximal value.
,Boolean indexing return all values in the array for which the index is True.
- Example:
index = a > 0.0
b = a[index]
print( b )
as mentioned before, when you want the number of elements in an array, you use the size()
function. You don’t use len() because this can sometimes be misleading
- Example:
A = [[1, 3], [2, 4]]
Len(a) returns 2
a.size returns 4
Week 2
Bookshelf analogy for multi-dimensional arrays:
1d array is a single row of a bookshelf, where a book can be identified by its position
in the row.
2d array is the whole bookshelf, where a book can be identified by its row number
and its position in the row.
There has to be the same number of books on every shelf.
3d array is a room full of bookshelves, where a book can be identified by the number
of the bookshelf, row, and position in the row.
4d array is a library with rooms with bookshelves, where a book can be identified by
the room, bookshelf, row, and position in the row.
...and so on...
Attributes to gain insight about the dimensionality:
- X.ndim number of dimensions
- X.shape length of each dimension
Can for example be (2, 2, 3) fist 2 tells you how many lists there are in the
array; second 2 tells you how many small lists are in the bigger lists; and 3 tells
you the size of each separate list.
Looks like:
[[[ 1 3 1]
[ 2 4 2]]
[[11 13 3]
[12 14 4]]]
- X.size total number of elements
An empty array or an array with only zeros and ones can be created with the empty, zeros
and ones functions.
One-dimensional empty array numpy.empty( (3) )
Two-dimensional array with zero’s numpy.zeros( (2, 3) )
Four-dimensional array with ones numpy.ones( (2, 2, 2, 2) )
, Identity matrix
= two-dimensional square matrix in which all values are zeros, except for the ones along the
diagonal.
This can be created with the eye(n) function.
- Example: numpy.eye(4)
Output:
[[1. 0. 0. 0.]
[0. 1. 0. 0.]
[0. 0. 1. 0.]
[0. 0. 0. 1.]]
More complex arrays can be created by combining two or more one-dimensional arrays.
- This is called vector stacking.
Horizontal stack:
- Numpy.hstack([x, y, z])
Example:
x = numpy.arange(0, 5)
y = numpy.arange(5, 10)
z = numpy.arange(10, 15)
numpy.hstack([x, y, z])
output = [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14]
shape = (15, )
Vertical stack:
- Numpy.vstack([x, y, z])
Example:
numpy.vstack([x, y, z])
output =
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]]
Shape = (3, 5)
You could also split arrays into a list of one-dimensional arrays (vectors).
Horizontal split:
- Numpy.hsplit(x, N)
Example:
a = numpy.arange(0, 5)
b = numpy.arange(5, 10)
c = numpy.arange(10, 15)
d = numpy.vstack([a, b, c])
numpy.hsplit(d, 5) d is the array you want to split
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through EFT, credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying this summary from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller sophiedekkers54. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy this summary for R137,60. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
85651 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy summaries for 14 years now