Skip to main content

Explaining your PostgreSQL data

I’ve written a bit before about understanding the output from EXPLAIN and EXPLAIN ANALYZE in PostgreSQL. Though understandably getting a grasp on execution plans could probably use some more guidance. Yet, this time around I’m taking a bit of a cop out and highlighting a few tools instead of documenting myself, which I’ve done in a talk I’ve frequently given Postgres Demystified.

Explain explained

Postgres Indexing - A collection of indexing tips

Even from intial reviews of my previous post on expression based indexes I received a lot of questions and feedback around many different parts of indexing in Postgres. Here’s a mixed collection of valuable tips and guides around much of that.

Unused Indexes

In an earlier tweet I joked about some SQL that would generate the SQL to add an index to every column:

# SELECT 'CREATE INDEX idx_' 
    || table_name || '_' 
    || column_name || ' ON ' 
    || table_name || ' ("' 
    || column_name || '");' 
  FROM information_schema.columns;
                      ?column?
---------------------------------------------------------------------
 CREATE INDEX idx_pg_proc_proname ON pg_proc ("proname");
 CREATE INDEX idx_pg_proc_pronamespace ON pg_proc ("pronamespace");
 CREATE INDEX idx_pg_proc_proowner ON pg_proc ("proowner");

The reasoning behind this is guessing whether an index will be helpful can be a bit hard within Postgres. So the easy solution is to add indexes to everything, then just observe if they’re being used. Of course you want to add it to all tables/columns because you never know if core of Postgres may be missing some needed ones

Postgres Indexes – Expression/Functional Indexing

Postgres is rich with options for indexing. First you’ve got a variety of types, and beyond that you can do a variety of things with each of these such as create unique indexes, use conditions to index only a portion of your data, or create indexes based on complex expressions or functions. In cases where you commonly use various PostgreSQL functions in your application or reporting you can get some great gains from this.

My SQL Bad Habits

I’m reasonably proficient at SQL – a coworker when pseudocoding some logic for him pointed out that my pseudocode is what he thought was executable SQL. I’m fully capable of writing clear and readable SQL – which most SQL is not. Despite that I still have several bad habits when it comes to SQL. Without further adieu heres some of my dirty laundry so hopefully others can not make the same mistakes.

CX – Conference Experience (Facilitating Communication)

Following up on my earlier post about CX or Conference Experience – I’m going to dig in a bit on how you get good conversation to happen. In the past two years I’ve been to nearly 20 conferences, I’ve been to conferences with great talks, with great parties, with great swag, and hands down my favorite conferences have always been a result of great conversation. With the number of talks that are recorded and immediately available online after, what can I say I’m a hallway track guy.

I’ve seen a number of conferences intentionally design around this concept, in some ways the unconference is purely a hallway track conference. I’ve also seen conferences that weren’t clearly planned for this and have pulled off some of the best situations where people turn their phones off and engage in real conversation.

CX – Conference Experience (Talks)

A couple of weekends ago I had the great opportunity to attend lessconf. It was an all around great conference, and as a result of the greatness I ended up having a conversation with a few people around conference experience. I must give much of the credit to Swift, as he mentioned he’d already been thinking alot about this since Waza. In general it feels like there’s a few key themes that any conference should focus on, then a lot of small things that can really push it over the top. Here’s a few:

In general the key areas for any great conference are:

  • Talks
  • Great people
  • Ensuring communication happens
  • Bonus points

Digging in deeper on the first area…

Heroku’s Acquisition 2 Years Later

Just over two years ago, Heroku was acquired. I was around and peripheral to this just before the acquisition and came on board only barely after. While there is a large group of people that have been there longer than I have (several for 4+ years) I’m still commonly asked how things have changed, how things work, and other questions of that nature. I wrote about some of these processes over a year ago in the months after joining Heroku around our hiring, our teams, and how we work. Many of these things haven’t changed, and yet almost always at a conference I’m asked how are things different since being acquired.

Here’s my personal take (and while I don’t typically include this – to be safe, this is not an official Heroku view of what’s changed).

Using array_agg in Postgres – powerful and flexible

In almost any application it’s common to want to aggregate some set of values together, commonly in a comma separated form. Most developers do this by running a query to get much of the raw data, looping over the data and pushing it into a set, appending each new value to the appropriate key. Hopefully, it’s not a surprise that there’s a much better way to do this with PostgreSQL.

Postgres has a flexible and robust array datatype that comes with a variety of functions. Even without taking advantage of the array datatype in your application, you can still take advantage of some of the functions to get the functionality you need. Lets take a look at an example schema and use case.

Scaling Evangelism – Creating Advocates

The first area I tend to think about when it comes to developer marketing is around advocates. A simple definition of an advocate is someone that speaks/writes in favor of some thing.

Creating advocates is a means of creating more of myself to go out and talk loudly and ideally effectively. However, they do take time to cultivate and it is a hard item to track. But by cultivating advocates I’m able to scale what would otherwise be a bottleneck of my own personal time.

Doing Marketing (for developers) Differently

As developers we can tend to be a fickle bunch; especially in certain open source communities. We like to see intelligence and systems applied to things, hope for a better world through making things open, and appreciate when others relate to our world as we often attempt to relate to theirs. Marketing is often a loaded word when it comes to developers – leaving mixed feelings about webinars, email campaigns, and the like.

A quick clarification in terms of marketing I’m referring to product marketing, but of a technical product. This is based on the idea that developers are the new kingmakers and when marketing a product to them it needs to be done differently.