Postgresql Online Tutorials

Interfaces edit

PostgreSQL supports a binary communication protocol that allows applications to connect to the database server. The protocol is versioned (currently 3.0, as of PostgreSQL 7.4) and has a detailed specification.

The official client implementation of this communication protocol is a C API, libpq. In addition, the officially supported ECPG tool allows SQL commands to be embedded in C code. Both are part of the standard PostgreSQL distribution.

Third-party libraries for connecting to PostgreSQL are available for many programming languages, including C++, Java, Julia, Python, Node.js, Go, and Rust.

Procedural languages edit

Procedural languages allow developers to extend the database with custom subroutines (functions), often called stored procedures. These functions can be used to build database triggers (functions invoked on modification of certain data) and custom data types and aggregate functions. Procedural languages can also be invoked without defining a function, using a DO command at SQL level.

Languages are divided into two groups: Procedures written in safe languages are sandboxed and can be safely created and used by any user. Procedures written in unsafe languages can only be created by superusers, because they allow bypassing a database's security restrictions, but can also access sources external to the database. Some languages like Perl provide both safe and unsafe versions.

PostgreSQL has built-in support for three procedural languages:

• Plain SQL (safe). Simpler SQL functions can get expanded inline into the calling (SQL) query, which saves function call overhead and allows the query optimizer to "see inside" the function.
• Procedural Language/PostgreSQL (PL/pgSQL) (safe), which resembles Oracle's Procedural Language for SQL (PL/SQL) procedural language and SQL/Persistent Stored Modules (SQL/PSM).
• C (unsafe), which allows loading one or more custom shared library into the database. Functions written in C offer the best performance, but bugs in code can crash and potentially corrupt the database. Most built-in functions are written in C.

In addition, PostgreSQL allows procedural languages to be loaded into the database through extensions. Three language extensions are included with PostgreSQL to support Perl, Tcl, and Python. For Python, the current Python 3 is used, and the discontinued Python 2 is no longer supported as of PostgreSQL 15. Both were supported previously, defaulting to Python 2, while old and new versions couldn't be used in the same session. External projects provide support for many other languages, including PL/Java, JavaScript (PL/V8), PL/Julia, PL/R, PL/Ruby, and others.

Triggers edit

Triggers are events triggered by the action of SQL data manipulation language (DML) statements. For example, an INSERT statement might activate a trigger that checks if the values of the statement are valid. Most triggers are only activated by either INSERT or UPDATE statements.

Triggers are fully supported and can be attached to tables. Triggers can be per-column and conditional, in that UPDATE triggers can target specific columns of a table, and triggers can be told to execute under a set of conditions as specified in the trigger's WHERE clause. Triggers can be attached to views by using the INSTEAD OF condition. Multiple triggers are fired in alphabetical order. In addition to calling functions written in the native PL/pgSQL, triggers can also invoke functions written in other languages like PL/Python or PL/Perl.

Asynchronous notifications edit

PostgreSQL provides an asynchronous messaging system that is accessed through the NOTIFY, LISTEN and UNLISTEN commands. A session can issue a NOTIFY command, along with the user-specified channel and an optional payload, to mark a particular event occurring. Other sessions are able to detect these events by issuing a LISTEN command, which can listen to a particular channel. This functionality can be used for a wide variety of purposes, such as letting other sessions know when a table has updated or for separate applications to detect when a particular action has been performed. Such a system prevents the need for continuous polling by applications to see if anything has yet changed, and reducing unnecessary overhead. Notifications are fully transactional, in that messages are not sent until the transaction they were sent from is committed. This eliminates the problem of messages being sent for an action being performed which is then rolled back.

Many connectors for PostgreSQL provide support for this notification system (including libpq, JDBC, Npgsql, psycopg and node.js) so it can be used by external applications.

PostgreSQL can act as an effective, persistent "pub/sub" server or job server by combining LISTEN with FOR UPDATE SKIP LOCKED.

Rules edit

Rules allow the "query tree" of an incoming query to be rewritten; they are an, automatically invoked, macro language for SQL. "Query Re-Write Rules" are attached to a table/class and "Re-Write" the incoming DML (select, insert, update, and/or delete) into one or more queries that either replace the original DML statement or execute in addition to it. Query Re-Write occurs after DML statement parsing and before query planning.

The functionality rules provide was, in almost every way, later duplicated with the introduction of newer types of triggers. The use of triggers is usually preferred over rules as it is easier to reason about trigger behavior and interactions than when equivalent rules are used.

Other querying features edit

• Transactions
• Full-text search
• Views
  • Materialized views
  • Updateable views
  • Recursive views
• Inner, outer (full, left, and right), and cross joins
• Sub-selects
  • Correlated sub-queries
• Regular expressions
• Common table expressions and writable common table expressions
• Encrypted connections via Transport Layer Security (TLS); current versions do not use vulnerable SSL, even with that configuration option
• Domains
• Savepoints
• Two-phase commit
• The Oversized-Attribute Storage Technique (TOAST) is used to transparently store large table attributes (such as big MIME attachments or XML messages) in a separate area, with automatic compression.
• Embedded SQL is implemented using preprocessor. SQL code is first written embedded into C code. Then code is run through ECPG preprocessor, which replaces SQL with calls to code library. Then code can be compiled using a C compiler. Embedding works also with C++ but it does not recognize all C++ constructs.

Concurrency model edit

PostgreSQL server is process-based (not threaded), and uses one operating system process per database session. Multiple sessions are automatically spread across all available CPUs by the operating system. Many types of queries can also be parallelized across multiple background worker processes, taking advantage of multiple CPUs or cores. Client applications can use threads and create multiple database connections from each thread.

PostgreSQL manages its internal security on a per-role basis. A role is generally regarded to be a user (a role that can log in), or a group (a role of which other roles are members). Permissions can be granted or revoked on any object down to the column level, and can allow or prevent the visibility/creation/alteration/deletion of objects at the database, schema, table, and row levels.

PostgreSQL's SECURITY LABEL feature (extension to SQL standards), allows for additional security; with a bundled loadable module that supports label-based mandatory access control (MAC) based on Security-Enhanced Linux (SELinux) security policy.

PostgreSQL natively supports a broad number of external authentication mechanisms, including:

• Password: either SCRAM-SHA-256, MD5 or plain-text
• Generic Security Services Application Program Interface (GSSAPI)
• Security Support Provider Interface (SSPI)
• Kerberos
• ident (maps O/S user-name as provided by an ident server to database user-name)
• Peer (maps local user name to database user name)
• Lightweight Directory Access Protocol (LDAP)
  • Active Directory (AD)
• RADIUS
• Certificate
• Pluggable authentication module (PAM)

The GSSAPI, SSPI, Kerberos, peer, ident and certificate methods can also use a specified "map" file that lists which users matched by that authentication system are allowed to connect as a specific database user.

These methods are specified in the cluster's host-based authentication configuration file (pg_hba.conf), which determines what connections are allowed. This allows control over which user can connect to which database, where they can connect from (IP address, IP address range, domain socket), which authentication system will be enforced, and whether the connection must use Transport Layer Security (TLS).

PostgreSQL claims high, but not complete, conformance with the latest SQL standard ("as of the version 15 release in October 2022, PostgreSQL conforms to at least 170 of the 179 mandatory features for SQL:2016 Core conformance", and no other databases fully conformed to it). One exception is the handling of unquoted identifiers like table or column names. In PostgreSQL they are folded, internally, to lower case characters whereas the standard says that unquoted identifiers should be folded to upper case. Thus, Foo should be equivalent to FOO not foo according to the standard. Other shortcomings concern the absence of temporal tables allowing automatic logging of row versions during transactions with the possibility of browsing in time (FOR SYSTEM TIME predicate), although relatively SQL compliant third-party extensions are available.

Many informal performance studies of PostgreSQL have been done. Performance improvements aimed at improving scalability began heavily with version 8.1. Simple benchmarks between version 8.0 and version 8.4 showed that the latter was more than ten times faster on read-only workloads and at least 7.5 times faster on both read and write workloads.

The first industry-standard and peer-validated benchmark was completed in June 2007, using the Sun Java System Application Server (proprietary version of GlassFish) 9.0 Platform Edition, UltraSPARC T1-based Sun Fire server and PostgreSQL 8.2. This result of 778.14 SPECjAppServer2004 JOPS@Standard compares favourably with the 874 JOPS@Standard with Oracle 10 on an Itanium-based HP-UX system.

In August 2007, Sun submitted an improved benchmark score of 813.73 SPECjAppServer2004 JOPS@Standard. With the system under test at a reduced price, the price/performance improved from $84.98/JOPS to $70.57/JOPS.

The default configuration of PostgreSQL uses only a small amount of dedicated memory for performance-critical purposes such as caching database blocks and sorting. This limitation is primarily because older operating systems required kernel changes to allow allocating large blocks of shared memory. PostgreSQL.org provides advice on basic recommended performance practice in a wiki.^[85]

In April 2012, Robert Haas of EnterpriseDB demonstrated PostgreSQL 9.2's linear CPU scalability using a server with 64 cores.^[86]

Matloob Khushi performed benchmarking between PostgreSQL 9.0 and MySQL 5.6.15 for their ability to process genomic data. In his performance analysis he found that PostgreSQL extracts overlapping genomic regions eight times faster than MySQL using two datasets of 80,000 each forming random human DNA regions. Insertion and data uploads in PostgreSQL were also better, although general searching ability of both databases was almost equivalent.^[87]

PostgreSQL is available for the following operating systems: Linux (all recent distributions), 64-bit ARM and x86-64 installers available and tested for macOS (OS X)^[88] version 10.12 and newer, Windows (with installers available and tested for 64-bit Windows Server 2019 and 2016; some older PostgreSQL versions were tested back to Windows 2012 R2^[89]), FreeBSD, OpenBSD,^[90] NetBSD, and these without official (though unofficial likely available) binary executables, Solaris,^[91] AIX, and HP-UX.^[92] Most other (modern) Unix-like systems do also work.

PostgreSQL can be expected to work on any of the following instruction set architectures (and operating systems): 64-bit x86-64 and 32-bit x86 on Windows XP (or later) and other operating systems; these are supported on other than Windows: 64-bit ARM^[93] and the older 32-bit ARM, including older such as ARMv6 in Raspberry Pi^[94]), z/Architecture, S/390, PowerPC (incl. 64-bit Power ISA), SPARC (also 64-bit), IA-64 Itanium (HP-UX), MIPS and PA-RISC. It was also known to work on some other platforms (while not been tested on for years, i.e. for latest versions).^[95]

Open source front-ends and tools for administering PostgreSQL include:

psql

The primary front-end for PostgreSQL is the psql command-line program, which can be used to enter SQL queries directly, or execute them from a file. In addition, psql provides a number of meta-commands and various shell-like features to facilitate writing scripts and automating a wide variety of tasks; for example tab completion of object names and SQL syntax.

pgAdmin

The pgAdmin package is a free and open-source graphical user interface (GUI) administration tool for PostgreSQL, which is supported on many computer platforms.^[96] The program is available in more than a dozen languages. The first prototype, named pgManager, was written for PostgreSQL 6.3.2 from 1998, and rewritten and released as pgAdmin under the GNU General Public License (GPL) in later months. The second incarnation (named pgAdmin II) was a complete rewrite, first released on January 16, 2002. The third version, pgAdmin III, was originally released under the Artistic License and then released under the same license as PostgreSQL. Unlike prior versions that were written in Visual Basic, pgAdmin III is written in C++, using the wxWidgets^[97] framework allowing it to run on most common operating systems. The query tool includes a scripting language called pgScript for supporting admin and development tasks. In December 2014, Dave Page, the pgAdmin project founder and primary developer,^[98] announced that with the shift towards web-based models, work has begun on pgAdmin 4 with the aim to facilitate cloud deployments.^[99] In 2016, pgAdmin 4 was released. The pgAdmin 4 backend was written in Python, using Flask and the Qt framework.^[100]

phpPgAdmin

phpPgAdmin is a web-based administration tool for PostgreSQL written in PHP and based on the popular phpMyAdmin interface originally written for MySQL administration.^[101]

PostgreSQL Studio

PostgreSQL Studio allows users to perform essential PostgreSQL database development tasks from a web-based console. PostgreSQL Studio allows users to work with cloud databases without the need to open firewalls.^[102]

TeamPostgreSQL

AJAX/JavaScript-driven web interface for PostgreSQL. Allows browsing, maintaining and creating data and database objects via a web browser. The interface offers tabbed SQL editor with autocompletion, row editing widgets, click-through foreign key navigation between rows and tables, favorites management for commonly used scripts, among other features. Supports SSH for both the web interface and the database connections. Installers are available for Windows, Macintosh, and Linux, and a simple cross-platform archive that runs from a script.^[103]

LibreOffice, OpenOffice.org

LibreOffice and OpenOffice.org Base can be used as a front-end for PostgreSQL.^[104][105]

pgBadger

The pgBadger PostgreSQL log analyzer generates detailed reports from a PostgreSQL log file.^[106]

pgDevOps

pgDevOps is a suite of web tools to install & manage multiple PostgreSQL versions, extensions, and community components, develop SQL queries, monitor running databases and find performance problems.^[107]

Adminer

Adminer is a simple web-based administration tool for PostgreSQL and others, written in PHP.

pgBackRest

pgBackRest is a backup and restore tool for PostgreSQL that provides support for full, differential, and incremental backups.^[108]

pgaudit

pgaudit is a PostgreSQL extension that provides detailed session and/or object audit logging via the standard logging facility provided by PostgreSQL.^[109]

wal-e

Wal-e is a backup and restore tool for PostgreSQL that provides support for physical (WAL based) backups, written in Python^[110]

A number of companies offer proprietary tools for PostgreSQL. They often consist of a universal core that is adapted for various specific database products. These tools mostly share the administration features with the open source tools but offer improvements in data modeling, importing, exporting or reporting.

Notable organizations and products that use PostgreSQL as the primary database include:

• Microsoft, used for a petabyte-scale “Release Quality View” (RQV) analytics dashboard, which tracks quality of Windows updates analyzing 20K types of metrics from over 800M Windows devices.^[111]
• In 2009, the social-networking website Myspace used Aster Data Systems's nCluster database for data warehousing, which was built on unmodified PostgreSQL.^[112][113]
• Geni.com uses PostgreSQL for their main genealogy database.^[114]
• OpenStreetMap, a collaborative project to create a free editable map of the world.^[115]
• Afilias, domain registries for .org, .info and others.^[116][117]
• Sony Online multiplayer online games.^[118]
• BASF, shopping platform for their agribusiness portal.^[119]
• Reddit social news website.^[120]
• Skype VoIP application, central business databases.^[121]
• Sun xVM, Sun's virtualization and datacenter automation suite.^[122]
• MusicBrainz, open online music encyclopedia.^[123]
• The International Space Station – to collect telemetry data in orbit and replicate it to the ground.^[124]
• MyYearbook social-networking site.^[125]
• Instagram, a mobile photo-sharing service.^[126]
• Disqus, an online discussion and commenting service.^[127]
• TripAdvisor, travel-information website of mostly user-generated content.^[128]
• Yandex, a Russian internet company switched its Yandex.Mail service from Oracle to Postgres.^[129]
• Amazon Redshift, part of AWS, a columnar online analytical processing (OLAP) system based on ParAccel's Postgres modifications.
• National Oceanic and Atmospheric Administration's (NOAA) National Weather Service (NWS), Interactive Forecast Preparation System (IFPS), a system that integrates data from the NEXRAD weather radars, surface, and hydrology systems to build detailed localized forecast models.^[117][130]
• United Kingdom's national weather service, Met Office, has begun swapping Oracle for PostgreSQL in a strategy to deploy more open source technology.^[130][131]
• WhitePages.com had been using Oracle and MySQL, but when it came to moving its core directories in-house, it turned to PostgreSQL. Because WhitePages.com needs to combine large sets of data from multiple sources, PostgreSQL's ability to load and index data at high rates was a key to its decision to use PostgreSQL.^[117]
• FlightAware, a flight tracking website.^[132]
• Grofers, an online grocery delivery service.^[133]
• The Guardian migrated from MongoDB to PostgreSQL in 2018.^[134]
• YugabyteDB implements the PostgreSQL query layer as its default SQL mode
• OpenAI uses PostgreSQL as part of its primary API service.^[135]

Some notable vendors offer PostgreSQL as software as a service:

• Heroku, a platform as a service provider, has supported PostgreSQL since the start in 2007.^[136] They offer value-add features like full database roll-back (ability to restore a database from any specified time),^[137] which is based on WAL-E, open-source software developed by Heroku.^[138]
• In January 2012, EnterpriseDB released a cloud version of both PostgreSQL and their own proprietary Postgres Plus Advanced Server with automated provisioning for failover, replication, load-balancing, and scaling. It runs on Amazon Web Services.^[139] Since 2015, Postgres Advanced Server has been offered as ApsaraDB for PPAS, a relational database as a service on Alibaba Cloud.^[140]
• VMware has offered vFabric Postgres (also termed vPostgres^[141]) for private clouds on VMware vSphere since May 2012.^[142] The company announced End of Availability (EOA) of the product in 2014.^[143]
• In November 2013, Amazon Web Services announced the addition of PostgreSQL to their Relational Database Service offering.^[144][145]
• In November 2016, Amazon Web Services announced the addition of PostgreSQL compatibility to their cloud-native Amazon Aurora managed database offering.^[146]
• In May 2017, Microsoft Azure announced Azure Databases for PostgreSQL.^[147]
• In May 2019, Alibaba Cloud announced PolarDB for PostgreSQL.^[148]
• Jelastic Multicloud Platform as a Service has provided container-based PostgreSQL support since 2011. It also offers automated asynchronous master-slave replication of PostgreSQL.^[149]
• In June 2019, IBM Cloud announced IBM Cloud Hyper Protect DBaaS for PostgreSQL.^[150]
• In September 2020, Crunchy Data announced Crunchy Bridge.^[151]
• In June 2022, Neon.tech announced Neon Serverless Postgres.^[152]

• Comparison of relational database management systems
• Database scalability
• List of databases using MVCC
• LLVM (llvmjit is the JIT engine used by PostgreSQL)