Let us get an overview of database programming.
-
When we develop web or mobile applications, data is typically stored in RDBMS Databases such as Oracle, Postgres, Mysql, etc.
-
Even data for reporting and analytics is typically stored in Data Warehouse or MPP Databases such as Redshift, Snowflake, Teradata, etc.
-
Here are different varieties of Databases for different purposes.
-
RDBMS - for example: Oracle, Mysql, Postgres, Informix, MS SQL, DB2, etc.
-
Data Warehouse - for example: Teradata, Vertica, Greenplum, RedShift, Snowflake, etc.
-
NoSQL - for example: MongoDB, Cassandra, HBase, DynamoDB, etc.
-
Search-based - Elasticsearch, Solr, etc.
-
Graph-based - Neo4J
-
-
Almost all the databases provide JDBC driver to connect to the database from the application layer.
-
Web or Mobile Applications are typically developed using Programming Languages like Python, Scala, Java, etc.
-
Data Engineering is typically done by tools such as Informatica or Programming Languages such as Python.
-
We need to use the JDBC or ODBC driver provided to connect to the specific database wherever Databases are involved, irrespective of the database that is used.
-
Also, one should be familiar with underlying Database Concepts to be comfortable with Database programming.
-
Typically, Databases will be set up on remote servers and accessible using the following information:
- Server IP or DNS Alias on which Database Server is running.
- Port Number using which the Database Server is running.
- Username
- Password
- Database Name
-
We typically perform CRUD Operations against our databases for Web or Mobile Applications for the transactions.
-
For non-transactional use cases, we typically process data in batches either within Databases or offload to ETL tools or distributed processing engines.
As part of this module, we will primarily focus on CRUD operations.