Introduction to Python Pandas

Pandas is a fundamental library for data manipulation in Python! Here's a quick introduction with some examples to get you started:

What is pandas?

Pandas is a powerful open-source library for data analysis in Python. It provides high-performance, easy-to-use data structures and data manipulation tools for working with tabular data like spreadsheets and databases.

It offers a wide range of features to:

• Import data from various sources like CSV files, Excel spreadsheets, databases, and more.
• Clean and prepare data by handling missing values, duplicates, and other inconsistencies.
• Explore and analyze data using descriptive statistics, grouping, filtering, and sorting operations.
• Perform data transformations by creating new columns, modifying existing data, and reshaping the data structure.
• Visualize data using integration with libraries like Matplotlib and Seaborn for creating informative charts and graphs.

Why use pandas?

Some of the key programming activity that you can perform using pandas are listed below:

• Data Cleaning and Manipulation: Pandas offers functions to clean messy data, handle missing values, and reshape your data for analysis.
• Data Analysis: You can perform various data analysis tasks like calculating statistics, grouping data, and aggregating results.
• Data Visualization: Pandas integrates well with plotting libraries like Matplotlib and Seaborn for data visualization.

Core Data Structures:

• Series: A one-dimensional array labeled with data (similar to a spreadsheet column).
• DataFrame: A two-dimensional labeled data structure with rows and columns (like a spreadsheet).

Getting started with pandas:

• Installation: If you don't have pandas installed, use pip:

pip install pandas  

• Import pandas:

import pandas as pd

Example 1: Creating a Pandas Series

# Prepare data variable

data = [1, 2, 3, 4, 5]

# Creating a Series

my_series = pd.Series(data)

print(my_series)

Output:

0 1

1 2

2 3

3 4

4 5

dtype: int64

Example 2: Creating a Pandas DataFrame

# Prepare data in dictionaries format

data = {

    'Name': ['Kuntal', 'Sunit', 'Tapan', 'Amit'],

    'Age': [40, 38, 42, 44],

    'Bank': ['SBI', 'HDFC', 'ICICI', 'KOTAK']

}

# Create a DataFrame

df = pd.DataFrame(data)

# Print the DataFrame

print(df)

Output:

Name Age Bank 0 Kuntal 40 SBI 1 Sunit 38 HDFC 2 Tapan 42 ICICI 3 Amit 44 KOTAK

Recommended Article for you:

• Popular Pandas Operations with Examples

Python program to replace given words from a text file.

Python - Replace Word Program

Following Python program will replace a given word from a text file.

Example: Assume that we have a students.txt file with a following content in it:

This is a java programming class.
We will develop an Insurance Calculator using java.
Let's explore java syntax during today's lab.

Following program will replace java with python in the "students.txt" file.

# Python program to replace given words from a text file.

old_word = 'python'

new_word = 'java'

with open("students.txt", 'r') as file:

    contents = file.read()

    contents = contents.replace(old_word, new_word)

with open("students.txt", 'w') as file:

    file.write(contents)

print(f'Replaced "{old_word}" with "{new_word}" in students.txt.

Sample output:

Replaced "java" with "python" in students.txt.

Content of a students.txt file after execution of a program:

This is a python programming class.

We will develop an Insurance Calculator using python.
Let's explore python syntax during today's lab.

Python File - Word Search Program

This file management functions searches for a specific word in a text file.

• Reads the file line by line and checks for word presence.
• Prints whether the word was found or not.

# Open the file in read mode

file = open("students.txt", "r")

# Get the word to search from the user

word_to_search = input("Enter the word to search: ")

# Initialize a flag to indicate if the word is found

word_found = False

# Read the file line by line

for line in file:

    # Remove leading and trailing whitespace

    line = line.strip()

    # Split the line into words

    words = line.split()

    # Check if the word is present in the current line

    if word_to_search in words:

        word_found = True

        print("Word found")

        break  # Stop searching once the word is found

# If the word is not found, print a message

if not word_found:

    print("Word not found in the file.")

Output of program:

Enter the word to search: amit

Word found

Following is the content of students.txt file.

NAME AGE ADDRESS

amit 24 usa

sunit 26 canada

kuntal 25 india

bhavin 27 india

* * * * * 

Program Using User Defined Functions

def search_word(filename, word):

    try:

        with open(filename, "r") as f:

            found = False

            for line in f:

                if word in line.lower():

                    found = True

                    break

        if found:

            print(f"Word '{word}' found in {filename}")

        else:

            print(f"Word '{word}' not found in {filename}")

    except FileNotFoundError:

        print("File not found!")

if __name__ == "__main__":

    filename = input("Enter file name: ")

    word = input("Enter word to search: ")

    search_word(filename, word)

Output of program:

Enter file name: products.txt
Enter word to search: pen
Word 'pen' found in products.txt

Note: I have saved this python program and a products.txt file in same folder.

Content of products.txt file.

pen,10,50

pencil,20,25 

sketchbook,20,10

notebook,10,30

* * * * *

Python Libraries for Data Science

Useful Python Libraries for Data Science

Python has become the go-to language for data science due to its vast and accessible library ecosystem. These libraries cover various aspects of data analysis, from data manipulation and visualization to machine learning and deep learning. Here's a brief introduction to some key libraries:

Foundational Libraries:

• NumPy: This library provides the backbone for efficient numerical computation in Python. It offers multi-dimensional arrays, called "ndarrays," which are optimized for performing mathematical operations on large datasets.
• Pandas: This library builds on top of NumPy, offering powerful data structures like Series and DataFrames for data manipulation, cleaning, and analysis. It provides efficient tools for filtering, sorting, grouping, and merging data.
• SciPy: This library extends NumPy's functionality with advanced scientific routines, including statistical analysis, optimization, and signal processing.

Data Visualization Libraries:

• Matplotlib: This library is the standard for creating static and interactive data visualizations in Python. It offers a wide range of plot types, from simple line charts to complex heatmaps and bar charts.
• Seaborn: Built on top of Matplotlib, Seaborn focuses on statistical graphics and produces aesthetically pleasing and informative visualizations. It's ideal for exploratory data analysis and presenting insights.
• Plotly: This library allows you to create interactive visualizations that can be displayed online or embedded in web applications. It provides a user-friendly interface for customizing charts and dashboards.

Machine Learning Libraries:

• Scikit-learn: This is the most popular library for machine learning in Python. It offers a wide range of algorithms for tasks like classification, regression, clustering, and dimensionality reduction. Scikit-learn is known for its user-friendly interface and extensive documentation.
• TensorFlow: This powerful library is designed for building and deploying deep learning models. It uses computational graphs to define and execute complex neural networks. TensorFlow is popular for research and development of cutting-edge AI applications.
• PyTorch: Another leading deep learning library, PyTorch offers a more flexible and dynamic approach compared to TensorFlow. It allows for easier debugging and experimentation with model architectures.

Additional Libraries useful in web development:

• Scrapy: This library helps you extract data from websites, allowing you to analyze web content and gather information for your projects.
• BeautifulSoup: This library simplifies parsing HTML and XML documents, making it easier to extract specific data from web pages.

This is just a small selection of the many useful libraries available for data science in Python. Choosing the right ones depends on your specific needs and the types of data you are working with.

 * * * * *

Top 10 Python Libraries

Followings are 10 of the most popular and widely used Python libraries as on December 2023 and their key usages:

1. NumPy:

• Foundation for numerical computing in Python.
• Efficiently handles large, multi-dimensional arrays and matrices.
• Offers mathematical functions, linear algebra operations, and random number generation.
• Essential for scientific computing, data analysis, and machine learning.

2. Pandas:

• High-performance data analysis and manipulation tool.
• Provides DataFrame and Series data structures for working with tabular data.
• Enables data cleaning, transformation, aggregation, and visualization.
• Widely used in data science, finance, statistics, and social sciences.

3. Matplotlib:

• Comprehensive library for creating static, animated, and interactive visualizations.
• Offers a wide range of plot types, including line, scatter, bar, histogram, pie charts, and 3D plots.
• Highly customizable and integrates well with other libraries.

4. Scikit-learn:

• Versatile machine learning library with a user-friendly API.
• Includes a variety of algorithms for classification, regression, clustering, dimensionality reduction, model selection, and preprocessing.
• Built on NumPy and SciPy, making it efficient and scalable.

5. TensorFlow:

• Open-source platform for numerical computation and large-scale machine learning.
• Used for building and training neural networks, deep learning models, and other machine learning algorithms.
• Supports distributed training, model deployment, and mobile development.

6. Keras:

• High-level API for building and training neural networks, written in Python and capable of running on top of TensorFlow or other backends.
• Known for its user-friendliness and ease of experimentation.
• Widely used for rapid prototyping and research in deep learning.

7. Requests:

• User-friendly library for making HTTP requests in Python.
• Simplifying interactions with web services and APIs.
• Handles headers, cookies, sessions, and authentication.

8. Beautiful Soup:

• Powerful library for parsing HTML and XML documents.
• Extracting data from websites, web scraping, and data cleaning tasks.
• Handles malformed markup and offers a variety of navigation and search methods.

9. SQLAlchemy:

• Object-relational mapper (ORM) for working with databases in Python.
• Abstracts database interactions, allowing you to work with data in a Pythonic way.
• Supports a wide range of database systems.

10. Flask:

• Lightweight and flexible web framework for building web applications in Python.
• Easy to learn and use, making it suitable for both small and large projects.
• Offers a modular design and a variety of extensions for different functionalities.

< HOME >

Python Pandas Series MCQs

 

Python Programming MCQs

Pandas Series

10 important MCQs related to Python Pandas Series, along with their answers:

---

1. What is a Pandas Series?

(A) One-dimensional labelled array 
(B) Two-dimensional labelled array 
(C) Three-dimensional labelled array 
(D) None of the above

2. How do you access a value from a Pandas Series?

(A) series[index] 
(B) series.index[value] 
(C) series.values[index] 
(D) series.data[index]

3. How do you sort a Pandas Series?

(A) series.index() 
(B) series.sort_values() 
(C) series.ascending() 
(D) series.descending()

4. How do you create a Pandas Series?

(A) import pandas as pd
    s = pd.Series([1, 2, 3, 4, 5])
(B) import pandas as pd
    s = pd.Series(data=[1, 2, 3, 4, 5])
(C) import pandas as pd
    s = pd.Series(index=['a', 'b', 'c', 'd', 'e'], data=[1, 2, 3, 4, 5])
(D) All of the above

5. Which of the following is a valid way to create a slice from Pandas Series "s"?

(A) s[:3]
(B) s[2:]
(C) s[1:4]
(D) All of the above

6. How can you find the sum of a Pandas Series (assuming all elements are numbers) ?

(A) series.sum()
(B) series.mean()
(C) series.std()
(D) None of the above.

7. How can you plot a Pandas Series as a line graph?

(A) series.plot(kind=’pie’)
(B) series.plot(kind='line')
(C) series.plot(kind='bar')
(D) None of the above.

8. Predict output of following python pandas code:

import pandas as pd

series1 = pd.Series([1, 2, 3, 5, 8, 10])

print(series1[1]+series1[3])

(A) 7
(B) 4
(C) 5
(D) 13

9. How can you check if a Pandas Series has any NaN values?

(A) Check the Series' dtype.
(B) Use the isna() method.
(C) Use the notnull() method.
(D) Check the Series' length.

10. Predict output of following python code:

import pandas as pd

series1 = pd.Series([3, 2, 4])

print(series1-1)

(A) Syntax error
(B) 1
(C) -2
(D) 

0 2

1 1

2 3

dtype: int64

---

ANSWERS: Python Pandas Series MCQs:

1.  (A) One-dimensional labelled array

2.  (A) series[index]

3.  (B) series.sort_values()

4.  (D) All of the above

5.  (D) All of the above

6.  (A) series.sum()

7.  (B) series.plot(kind='line')

8.  (A) 7

9.  (B) Use the isna() method

10. (D)

0    2

1    1

2    3

dtype: int64

 

< Back to Python MCQs >

Python Pandas MCQs Set 1

 

MCQs based on Python Pandas

10 important MCQs related to Python Pandas, along with their answers:

---

1. What is the purpose of the pandas library?

(a) To perform data analysis and manipulation

(b) To create web applications

(c) To handle machine learning tasks

(d) To visualize data

2. What is the difference between a DataFrame and a Series in Pandas?

(a) A DataFrame is a two-dimensional data structure, while a Series is a one-dimensional data structure.

(b) A DataFrame can store data of different types, while a Series can only store data of a single type.

(c) A DataFrame is like an Excel worksheet, while a Series is like a column of an Excel worksheet.

(d) All of the above

3. What is the output of the following code?

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'], 'Age': [32, 25, 27]}

df = pd.DataFrame(data)

print(df['Age'].max())

(a) 27

(b) 25

(c) 32

(d) None of these

4. Which of the following is not a built-in data structure in Pandas?

(a) Series

(b) DataFrame

(c) Panel

(d) Numpy ndarray

5. What is the purpose of the head() function in Pandas?

(a) To print the first few rows of a DataFrame

(b) To sort a DataFrame by a specific column

(c) To calculate the mean of a DataFrame

(d) To create a new DataFrame from an existing DataFrame

6. What is the purpose of the tail() function in Pandas?

(a) To print the first few rows of a DataFrame

(b) To sort a DataFrame by a specific column

(c) To calculate the mean of a DataFrame

(d) To create a new DataFrame from an existing DataFrame

7. What is the purpose of the dropna() function in Pandas?

(a) To print the first few rows of a DataFrame

(b) To sort a DataFrame by a specific column

(c) To calculate summary statistics for a DataFrame

(d) To remove rows from a DataFrame that contain missing values

8. What is the purpose of the fillna() function in Pandas?

(a) To print the first few rows of a DataFrame

(b) To sort a DataFrame by a specific column

(c) To calculate summary statistics for a DataFrame

(d) To replace missing values in a DataFrame

9. What is the output of the following python pandas code?

import pandas as pd

data = {'Name': ['Sunit', 'Bob', 'Amit'], 'Age': [32, 25, 27]}

df = pd.DataFrame(data)

print(df['Age'].mean())

(a) 32

(b) 28.0

(c) 25

(d) 27

10. How do you access a specific column in a pandas DataFrame?

(a) df[column_name]

(b) df.column_name

(c) df.get_column(column_name)

(d) df['column_name']

---

Python Pandas MCQs Answers:

1. (a) To perform data analysis and manipulation

2. (d) All of the above

3. (c) 32

4. (d) Numpy ndarray

5. (a) To print the first few rows of a DataFrame

6. (d) To print the last few rows of a DataFrame

7. (d) To remove rows from a DataFrame that contain missing values

8. (d) To replace missing values in a DataFrame

9. (b) 28.0

10.(d) df['column_name']

 

< Back to Python MCQs >

Python Basics MCQs Set 2

MCQs based on Python Basics (Set 2)

10 important MCQs related to Python basic syntax, along with their answers:

---

1. Which of the following is the correct syntax for assigning a value to a variable in Python?

(a) my_variable := 10

(b) my_variable = 10

(c) my_variable = "10"

(d) my_variable '10'

2. What is the output of the following Python code?

print("Hello, World!)

(a) Hello, World!

(b) "Hello, World!

(c) Syntax Error

(d) Import Error

3. What is the data type of the variable my_number in the following Python code?

my_number = 10

(a) int

(b) float

(c) string

(d) boolean

4. Which of the following is the correct syntax for an if statement in Python?

(a) if condition:

      body

(b) if condition;

      body

(c) if condition: {

      body }

(d) if condition; {

      body }

5. What is the difference between the + and = operators in Python?

(a) The + operator is used for addition of numbers, while the = operator is used for assignment.

(b) The + operator is used for concatenation of numbers, while the = operator is used for assignment.

(c) The + operator is used for assignment while the = operator is used for addition.

(d) None of these

6. What is the purpose of the print() function in Python?

(a) To print output to the console

(b) To read input from the user

(c) To convert data types

(d) To check if a string is empty

7. What is the purpose of the indentation in Python code?

(a) To make the code more readable

(b) To define code blocks

(c) To prevent errors

(d) All of the above

8. What is the purpose of the for loop in Python?

(a) To prevent errors from occurring

(b) To check if a condition is true

(c) To handle errors in code

(d) To iterate over a sequence of items

9. Which of the following is not a valid Python identifier?

(a) my_variable

(b) 1st_name

(c) while1

(d) _my_variable

10. What is the purpose of the input() function in Python?

(a) To convert data types

(b) To print output to the console

(c) To read input from the user

(d) To check if a string is empty

Answers

1. (b) my_variable = 10

2. (c) Syntax Error

3. (a) int

4. (a) if condition:

         body

5. (a) The + operator is used for addition of numbers, while the = operator is used for assignment. 

6. (a) To print output to the console

7. (b) To define code blocks

8. (d) To iterate over a sequence of items

9. (b) 1st_name

10. (c) To read input from the user

< Back to Python MCQs >

Python String MCQs Set 1

MCQs based on Python String

---

1. Which of the following methods is used to convert a string to uppercase?

A.   upper()

B.    capitalize()

C.    title()

D.   all()

 

2. Which of the following methods is used to remove whitespace from the beginning and end of a string?

A.   lstrip()

B.    rstrip()

C.    strip()

D.   replace()

 

3. Which of the following methods is used to count the number of occurrences of a substring in a string?

A.   count()

B.    find()

C.    index()

D.   rfind()

 

4. Which of the following methods is used to split a string into a list of words?

A.   split()

B.    splitlines()

C.    partition()

D.   rsplit()

 

5. Which of the following methods is used to format a string with placeholder values?

A.   format()

B.    f-strings

C.    % operator

D.   str.format()

 

6. Which of the following is the correct syntax for replacing all occurrences of a substring within a string with another substring?

A.   string.replace(old, new)

B.    replace(string, old, new)

C.    string.replaceAll(old, new)

D.   string.replaceSubstring(old, new)

7. Which of the following is the correct syntax for splitting a string into a list of words based on spaces?

A.   string.split()

B.    split(string)

C.    string.splitWords()

D.   string.wordSplit()

 

---

Answers:

1. (A) upper()

2. (C) strip()

3. (A) count()

4. (A) split()

5. (D) str.format()

6. (A) string.replace(old, new)

7. (A) string.split()

< Back to Python MCQs >