Phoenix Framework REST API Crash Course - Introduction + Full Elixir API Tutorial

Timestamps

• 00:00 - Intro
• 01:20 - What is Elixir Phoenix
• 03:38 - MVC Pattern
• 05:49 - Installation
• 08:25 - Project Structure
• 10:03 - Main files
• 11:08 - Other Folders
• 12:12 - Create a Repo
• 13:59 - Basic Application Architecture
• 16:12 - Adding a New Page with Mock Data
• 22:24 - Creating an API
• 22:49 - Working with the Database
• 31:53 - Routes
• 32:48 - Updating a Resource
• 34:45 - Delete
• 35:23 - Users and Relationships
• 36:01 - Update the Model
• 37:23 - Making the association in models
• 40:25 - Nested Data in API Calls
• 43:17 - Get a User's Posts
• 48:21 - Conclusion

Elixir is the programming language, Phoenix is the web framework built on top of it.

Phoenix is pretty much always used for the back end, so before anything else it's worth being clear on what a back end even is. When you visit a website you're looking at the front end, the nice visual of the data. That data got fed to it from somewhere else, and that somewhere else is the back end. What looks like one application is usually at least two.

The back end takes data from a database (sqlite, Postgres, MySQL, plenty of others) and hands it to the front end in a predictable format. That format is JSON, which is just a way to represent data as text. A back end that lets you request certain information and get it back as JSON is an API.

By the end of this you'll have a working API that pulls from a database and returns JSON.

The prerequisite is my Elixir Programming Introduction. That one covers the language itself, this one is applying it to the web. All the code for this one is up at github.com/CalebCurry/elixir-phoenix if something isn't working on your machine and you want to check against mine.

MVC Pattern

Once an application gets bigger you need a way to organize the code, and there are a bunch of design patterns for that. The most common is MVC.

• Model - the data layer, talks to the database
• View - the presentation, what gets displayed
• Controller - sits between the two and does the actual logic

The flow is: a user hits the app, the request lands in a controller, the controller asks the model for data, the model goes to the database and hands it back, and then the controller renders a view which gets sent back to the user. Phoenix also puts a router in front of all that, which is the layer that associates a request path with a function in a controller.

In an API the view isn't an HTML page, it's the JSON structure. Same idea, different output. There's probably variations of this diagram all over the internet so I'm not saying this is the only way to think about it, but it's the pattern Phoenix recommends and the one we'll follow.

Installation

Phoenix ships as a Mix archive, so first you need Hex:

mix local.hex
mix archive.install hex phx_new

phx_new is what gives us the mix phx.new generator. The basic way to make a project is mix phx.new <name>, which assumes you want a database and that the database is Postgres. I want sqlite3 instead, it's just easier to get going with:

mix phx.new forum --database sqlite3

Accepted values for --database are postgres, mysql, mssql, and sqlite3 (the full option list is here). If you don't want a database at all, --no-ecto. I do want one.

Phoenix uses Ecto to talk to the database, which is an ORM. The nice part is the database you pick isn't a huge deal, if you're on Postgres and I'm on sqlite the application code looks pretty much the same, Ecto abstracts away the database specific stuff.

It'll ask to fetch and install dependencies, say yes. Then:

cd forum
mix ecto.create
mix phx.server

The default config is fine so mix ecto.create just works, and you'll see forum_dev.db show up. mix phx.server runs the app on port 4000, so command-click (or control-click on Windows) the address and you'll get the Phoenix landing page. That's your first Phoenix website. It's local and it's basically nothing, but it's a start.

If you want to change the port or anything else, that lives in config/dev.exs.

Project Structure

It generates a LOT of files, so here's where to find what.

First the hidden stuff, the directories starting with a dot or an underscore. You don't usually go in and tinker with these:

• _build - build artifacts, generated on the fly, safe to delete
• .elixir_ls - language server files, from the Elixir LS extension
• .gitignore - this one you DO edit, it's what keeps build artifacts out of your repo
• .formatter.exs - config for mix format

mix format formats your source code. Open a second terminal tab so you keep the server running, and run it there. I have the Elixir formatter extension doing this on save, so if you see my code jump around while I'm typing, that's what that is.

Main files

mix.exs is the project file. If you've worked with node this is basically your package.json, it describes the project and lists your dependencies. Scroll down and you'll see them. Add a new one to that list or change a version, then:

mix deps.get

You'll need that whenever you first clone a project from GitHub too, to make sure you actually have everything.

What gets retrieved is written down in mix.lock, which pins the exact versions. That matters more than it sounds, dependencies change over time and if you and another developer are on different versions the code might behave differently or not compile at all on one of your machines.

Other folders

• assets - front-end junk, pretty much ignore it for now
• config - I'll let you guess
• deps - the actual downloaded dependency code, gitignored, since anyone cloning can just run mix deps.get
• priv - accessory files that aren't directly running the app, scripts and our migrations live here
• test - tests, probably not going to need it here
• lib - our app code, this is the one you actually work in

Create a Repo

Something I like to do the moment a project is generated is commit the whole thing, so I have an initial commit that's a clean untouched application to revert back to. As you're learning I'd encourage the same:

git init
git add .
git status
git commit -m "init"

Then git log shows the history. I pushed mine up to GitHub as a public repo called elixir-phoenix so you can follow along commit by commit and see exactly what changed at each step.

Basic Application Architecture

We can already reach the default page, so let's trace how it gets there. Available paths live in the router, at lib/forum_web/router.ex:

scope "/", ForumWeb do
  pipe_through :browser

  get "/", PageController, :home
end

A get with just a slash means the homepage, and it uses PageController and the :home function. So follow it through to lib/forum_web/controllers/page_controller.ex:

def home(conn, _params) do
  render(conn, :home)
end

In the video this reads render(conn, :home, layout: false). Layouts got reworked in Phoenix 1.8, there's now just the one root.html.heex wrapping everything and the app layout is a plain function component you call from a template, so the generated controller doesn't pass layout: false anymore. The block above is the current version.

All it does is render. The page itself is under lib/forum_web/controllers/page_html/home.html.heex. There's an heex extension you can grab if you're not getting highlighting.

Open it and replace the whole thing (it's a pretty complex homepage) with:

<h1>Hello and welcome</h1>

Save, and the server rebuilds on its own, you don't have to stop and start it. Refresh and there it is.

If that felt like a lot of jumping around, it is. One more time: router takes the request and associates it with a controller, the controller does whatever application logic (here, nothing), it renders a view, the view goes back to the user. That's the MVC diagram exactly, minus the model, since we aren't touching data yet.

Adding a New Page with Mock Data

Now let's build that same structure ourselves, starting with hardcoded data. Same workflow, so start at the router:

scope "/", ForumWeb do
  pipe_through :browser

  get "/", PageController, :home
  get "/users", PageController, :users
end

We're reusing PageController here. On a bigger app you'd probably have a dedicated UserController instead.

Then the function in page_controller.ex:

def users(conn, _params) do
  IO.puts("users function hit")

  users = [
    %{id: 1, name: "Alice", email: "alice@email.com"},
    %{id: 2, name: "Bob", email: "bob@email.com"}
  ]

  render(conn, :users, users: users)
end

Same deal as the homepage, the video passes layout: false here and on 1.8 you don't. If you DO want the shared chrome (a header or footer across multiple pages) you wrap the template's contents in the app layout component instead, <Layouts.app flash={@flash}> ... </Layouts.app>.

The IO.puts is just so you can see something show up in the terminal, and users: users is how the data gets handed to the view. Then create lib/forum_web/controllers/page_html/users.html.heex:

<h1>Users page!</h1>
<ul>
    <%= for user <- @users do %>
    <li><%= user.name %> - <%= user.email %></li>
    <% end %>
</ul>

The template engine here is embedded Elixir, and all those <%= %> symbols are just how you mark where Elixir code starts and stops. Don't worry too much about the details, this is pretty much the last time I use it in this video, I don't usually spend a ton of time studying template engines. There are syntax guides if you want to go deeper.

Visit localhost:4000/users and you get Alice and Bob, and users function hit in the terminal. Right click, view page source, and it's pure HTML, no Elixir anywhere. The page got generated on the back end and the finished HTML got sent to the client.

That's one way to build full stack apps and it has real limits. You're stuck on a fairly niche templating engine, which is fine for something simple but not great next to a front-end framework like React, and by default every navigation is a full page refresh, so it's not friendly for single page applications. It IS worth knowing the back end can do either, you don't technically need two separate applications.

Creating an API

I want JSON instead, and that part is easy. Swap the render call out:

def users(conn, _params) do
  users = [
    %{id: 1, name: "Alice", email: "alice@email.com"},
    %{id: 2, name: "Bob", email: "bob@email.com"}
  ]

  json(conn, %{users: users})
end

users is the key, the list is the value. Visit the page now and you get JSON.

So we have a JSON endpoint, it's just mocked. Time for the database.

Working with the Database

The easiest way to start is to have Phoenix generate the code and then adapt it. Ignore the users stuff for a second, we're building a forum, so let's do posts:

mix phx.gen.json Posts Post posts title:string body:string

Three "posts" in a row is confusing so here's what each one is:

• Posts - the context, the module responsible for encapsulating data access. Think of this as the model part of MVC.
• Post - the schema, which describes what a single post looks like.
• posts - the plural, used for the table name.

That creates a pile of files and then prints a line to add to your router. I want the API under /api, so instead of pasting it where they suggest, make a new scope:

scope "/api", ForumWeb do
  pipe_through :api
  resources "/posts", PostController, except: [:new, :edit]
end

pipe_through :api is what lets us hang middleware off this later.

The next thing it tells you to do is apply your migrations. A migration describes a change to the database structure, and stacked up they show how you got from nothing to what you have now. The one that just got generated is under priv/repo/migrations/ and it's called something like create_posts. It describes the change but doesn't apply it:

mix ecto.migrate

Now with the server running, hit localhost:4000/api/posts and you get an empty list. That's exactly what you want to see. It's empty because we have no posts yet.

Creating a post

For this I'm using Postman. Maybe a bad choice of example, because the "post" in Postman has nothing to do with posts on a forum, it's referring to a POST request. Any API testing tool works if you already have one you like.

New request, method POST, URL http://localhost:4000/api/posts (note there's no s on the http, we're on localhost). Then body, raw, JSON:

{
  "post": {
    "title": "My first post",
    "body": "Let me tell you about myself"
  }
}

Hit send and you get the same thing back with an id of 1. The id comes from the database and acts as your confirmation that the insert happened. Refresh the GET in the browser and the data's there.

What the generated code is actually doing

Follow the router to lib/forum_web/controllers/post_controller.ex and look at create:

def create(conn, %{"post" => post_params}) do
  with {:ok, %Post{} = post} <- Posts.create_post(post_params) do
    conn
    |> put_status(:created)
    |> put_resp_header("location", ~p"/api/posts/#{post}")
    |> render(:show, post: post)
  end
end

That second parameter is the interesting one. It's pattern matching: take a map with a "post" key and assign the value to post_params. Which is exactly the shape of the JSON we sent.

If you want to see it, drop an IO.inspect(post_params) in there and watch the terminal.

The with line is doing the same trick on the result of Posts.create_post. Pull it apart so you can actually see what's being matched:

def create(conn, %{"post" => post_params}) do
  result = Posts.create_post(post_params)
  IO.inspect(result, label: "db result")

  with {:ok, %Post{} = post} <- result do
    conn
    |> put_status(:created)
    |> put_resp_header("location", ~p"/api/posts/#{post}")
    |> render(:show, post: post)
  end
end

Insert something and search the terminal for db result. It's a tuple, :ok first, then a Post struct. Now break it on purpose, change "title" to "tile" in the request body and send. You get can't be blank, and db result now has :error as the first value. That doesn't match the pattern, so the body of the with never runs.

Below that we're just building the response, put_status(:created) is where the 201 Created comes from, then the headers, then a render. That render still goes through a template, it just produces JSON. You'll find it in lib/forum_web/controllers/post_json.ex:

def show(%{post: post}) do
  %{data: data(post)}
end

Change data: to post: and the top level key in your response changes. Same for index. You can still think of post_json.ex as the view layer, it's just handing back JSON instead of HTML.

Routes

A lot of this got wired up by convention. To see everything you have:

mix phx.routes

Getting the list is index, getting one is show, then create, then update (twice, once for PATCH and once for PUT), then delete. The :id in a path is the part you pass in, so /api/posts/1 hits show and gets you that one post.

Updating a Resource

There are two methods for updating and they run the same function here. In general PUT replaces a resource and PATCH updates it, meaning with a PATCH anything you leave out keeps whatever it had.

In Phoenix, by default, both of them behave like a PATCH. Send a PUT to /api/posts/1 with just a title and the title changes, but the body stays exactly where it was. So it's not really a replace.

Because of that I'd use PATCH when I'm updating, just for clarity. If you want a real replace you could write a separate function for PUT where leaving out body actually removes it, feel free to do that as an exercise. I think the default behavior is fine.

Delete

This one's easy, which is about the only thing I'm good at. Send a DELETE to /api/posts/1 and you get back a 204 No Content, which is a pretty normal response for a delete since there's no body to give you. Send it again and you get a 404, because the thing you're trying to delete isn't there anymore.

That's your full CRUD: create, read, update, delete.

Users and Relationships

Next up, a user who owns posts. This is relationships, which gets a little more involved, so if what we've covered is already enough for what you need, no biggie. It's really not too bad though.

Same generator as before, different arguments:

mix phx.gen.json Accounts User users name:string email:string:unique

:unique on the email is an extra attribute type you can tack on. Right before running this I made a commit, which makes it very easy to see exactly what files the command added.

Add the line it prints to the API scope:

scope "/api", ForumWeb do
  pipe_through :api
  resources "/posts", PostController, except: [:new, :edit]
  resources "/users", UserController, except: [:new, :edit]
end

That mock users route from earlier was just for demonstration, I'm removing it. Then apply the new migration:

mix ecto.migrate

Update the model

We have a users table now, but nothing connects a post to a user. That needs a user_id column on posts, which means a new migration:

mix ecto.gen.migration add_user_id_to_posts

Open that file and it's empty, it has no idea what you want. Describe the change yourself:

def change do
  alter table(:posts) do
    add :user_id, references(:users, on_delete: :nothing)
  end
end

The alter table wrapper is the part that's easy to forget. Leave it off and mix ecto.migrate gives you cannot execute command outside of block, which is not the most obvious error message in the world. With it in place:

mix ecto.migrate

Making the association in models

That changed the database but not our code. Open lib/forum/posts/post.ex and there's no mention of a user anywhere, so the association needs made in two places, once on each schema.

Post belongs to a user:

schema "posts" do
  field :title, :string
  field :body, :string
  belongs_to :user, Forum.Accounts.User
  timestamps(type: :utc_datetime)
end

That's application, then context, then schema. And a user has many posts, over in lib/forum/accounts/user.ex:

schema "users" do
  field :name, :string
  field :email, :string
  has_many :posts, Forum.Posts.Post
  timestamps(type: :utc_datetime)
end

Nested Data in API Calls

To test this we need a user first. New request in Postman, POST to /api/users:

{
  "user": {
    "name": "Caleb",
    "email": "caleb@example.com"
  }
}

Send, and you get back an id of 1.

Now try to create a post owned by that user:

{
  "post": {
    "title": "My second post",
    "body": "Some content",
    "user_id": 1
  }
}

The post gets created but the user_id does NOT get attached. There's a structure in the database, sure, but there's also application level validation, and that's the changeset. Look at the bottom of post.ex, it only casts body and title. Add the new field:

def changeset(post, attrs) do
  post
  |> cast(attrs, [:body, :title, :user_id])
  |> validate_required([:body, :title, :user_id])
end

Send it again and it still doesn't show up in the response. But drop user_id out of the request now and you get can't be blank, which tells you it IS reaching the database, it's just not being displayed.

Displaying it is a view concern, so post_json.ex:

def data(%Post{} = post) do
  %{
    id: post.id,
    body: post.body,
    title: post.title,
    user_id: post.user_id
  }
end

Now it comes back. Fetch the earlier posts and you'll see the split: the one you made right after fixing the changeset has a user_id (it made it to the database, it just wasn't being returned), and the ones before that have nothing, because the value never got that far.

So there's two pieces here, the view decides what shows up in the API and the schema decides what's allowed through to the database. Neither one needs a migration, because we're changing how the application works, not the database structure.

Get a User's Posts

Request a user right now and the posts aren't included. Partly that's display, but there's one more thing first: the data has to be preloaded.

Related data doesn't get retrieved by default, on purpose. If all you want is the user's name it'd be a waste to go collect every post they've written, and on a complicated database with a lot of relationships you really don't want it wandering off into every other table on its own. So you say when you want it, in the context, at lib/forum/accounts.ex:

def get_user!(id), do: Repo.get!(User, id) |> Repo.preload(:posts)

Then the display side, in lib/forum_web/controllers/user_json.ex. You can't just stick user.posts in the map, a list of structs isn't something it can render, you'll get an undefined error. What you can do is copy the pattern that's already there and loop:

alias Forum.Posts.Post

defp data(%User{} = user) do
  %{
    id: user.id,
    name: user.name,
    email: user.email,
    posts: for(post <- user.posts, do: post_data(post))
  }
end

defp post_data(%Post{} = post) do
  %{
    id: post.id,
    title: post.title,
    body: post.body
  }
end

The alias is just so you don't have to write out the fully qualified name every time you use it. Get user 1 now and you get the user plus a nested list of their posts.

There's a cleaner way though. PostJSON already has a function that does exactly this shape, so instead of writing a second one, drop the defp down to def in post_json.ex so it's public, then reuse it:

alias ForumWeb.PostJSON

defp data(%User{} = user) do
  %{
    id: user.id,
    name: user.name,
    email: user.email,
    posts: Enum.map(user.posts, &PostJSON.data/1)
  }
end

Same result, and this version includes the user_id too, since that was the one difference between the function I wrote and the one that was already sitting there. Enum.map takes the list and a function, and &PostJSON.data/1 is the capture syntax for passing that function along.

That's a foundation. There's obviously a lot more you could do from here, the obvious next one being letting a user register and log in so they can only create posts as themselves, which is a decent thing to try implementing on your own.

This and the Elixir Programming Introduction are the first Elixir content I've done on YouTube, so I'm genuinely unsure what you all want next. Drop a comment and let me know.