#237 Solid Queue, Cache, Cable with Postgres in Production

Hello. So previously we deployed Mon Gun locally and set up Stripe to work correctly with webhooks so that we can actually subscribe organizations to real, uh, plans. And now we're going to deploy to production. And it is going to be a bit tricky because, uh, now we're using Postgre scale and we have, uh, slow Q, so cache and solid, uh, uh, cable installed by default. And, uh, they are set up initially to work with multiple SQI databases. So we are going to have to make them work with PostGresQ. So let's have a look at our database. I ml here you see we have a Postgre Scale set up the database because we initial generated the application by running Rails new, uh, gon, uh, database equals Postgre scale. So it set up a postgre scale as the main database. But if you go to our production, you see we also have, uh, uh, separate cache database, separate queue database, and a separate cable database. And, uh, we will want to use the same, the primary database for all these three. Now let's see if we have a, uh, sold cache, queue and cable installed correctly. Uh, to check this, we are going to run the install command for each of them. So I will, uh, run, uh, bin slash rails, uh, solid, uh, cable install and see if there is any diff. So let's go to, uh, gi. And you see there is a diff it added the cable schema. Let's also run the same command for solid, uh, uh, uh, queue, uh, run for solid queue cable and run for solid cache. And you see it added the three schemas. So the idea is that, uh, rails wants to create multiple databases, uh, the primary database and three databases for cable cache and q. And we don't want to have the three separate database. We want to have these tables all in the main, uh, schema in the same, uh, main database. So we are going to, uh, add this as migrations to our main database. I will run a command rails, uh, generate migration, uh, add solid, uh, cache, Queue and cable. Uh, now we have three empty, uh, migrations will go to DB migrate. So we have this add solid cache and they're going to copy the, uh, info from cache schema. They're going to copy this great table and um, add it here. So it's the solid cache fund. Uh, then we'll do the solid, uh, queue the of tables and indexes here. So adding solid queue and finally solid cable. Okay, now I'm going to delete the other three, uh, schemas. We will not need them. I can draw rails to B migrate and in database via ml. Here. I'm going to remove these three separate, uh, uh, database definitions for production. And here in primary, uh, I'm going to use the, just the primary database. We have primary setup here and, uh, I think in queue here. Uh, yeah, we're just using default in, um, Whether we have the cable. So here we will say that the writing is for database, uh, primary, but also I think in theory you can just command this out, uh, and it'll use primary default. And for cache, we are also going to say that the database is going to be not the cache database that we just removed from database via ml, but the primary database. Okay, uh, what's next? So we kind of set it up to use, uh, the main postgre scale database. Now, let's, uh, set up solid queue to work, uh, locally. So if go to production, uh, solid. So cto, solid C store, uh, Q adapter is solid queue and connects to database right in queue. We need to command this out. They're just, uh, going to use the main database. We can just comment it out. And, uh, we have the queue adapter, solid queue. Let's see what we have in development. Queue adapter. Um, it's not set at all 'cause it is in line by default. So the default value is inline. We don't have to actually state it, but we're not going to use inline. We're going to use, uh, sold queue as the, uh, queue adapter. Uh, now let's, uh, try adding a job and see if, uh, we can trigger the job. But, uh, before adding a job, let's see, let's add the, uh, dashboard to see jobs. Let's add the, I think it's called, uh, what's the name of the tool? Let me see. Mm. Mission control. Yes. Mission control jobs. A dashboard to see all our jobs for solid queue. So let's add the Gem Mission control jobs to our gem file. Let's add it next to the other solid gems 'cause it's, uh, directly related. Uh, now we'll run bundle And uh, we are going to mount the jobs, but we'll make it so that only admin, uh, user can see them. So we need an authenticated admin and they'll also have to disable the default basic HTP uh, authentication. So I will go to application rb and I'm going to disable this basic auth authentication because we already have this, uh, user authentication here for the jobs. So let's, uh, uh, run the server bin Dev. Yeah, I think we already have it running. I'll just restart it because we added a lot of changes in environments and we'll go to slash uh, jobs. It is not present 'cause the current user is not an admin. So let's make an an admin rail. C user first. Update Admin. True. We'll refer this page and you see we have access to the jobs. Okay, so we have installed mission control. Now let's add a job. Um, let's create a job named the Hello World. So Rails generate job. Hello world. Uh, let's open the job. Let's say puts Hello World and let's, uh, uh, schedule this Job Rails console. Uh, hello, job, the job perform later. Let's go to our jobs dashboard and see we have one Penant job that, uh, needs to be performed and it hasn't been performed yet. Uh, why? Because we don't have, uh, so queue running. So we need to run, uh, solid queue. Let's see how to do it. Uh, there is it in the docks, maybe have it noted somewhere. Let's see. Ah, it should be stored. The command is, uh, sold to start. I think finally I can find it in the box. Let's try manually running, uh, rails. Solid queue start. We'll do anything. Yes, you see it, uh, pinned. Hello world. And if we go back to our dashboard, you see there are no pending jobs. We have, uh, one unfinished jobs. Hello will job. So, uh, we managed to start solid queue and, uh, it processed the job. We can move this to our profile Profile dev. So I will say Worker is run by sold q start Now. We don't need this terminal anymore. I will restart the main pin dev to also include the worker and the let me schedule another job performed later. You see we have two jobs that have been performed. Uh, it has been performed instantly, instantly. So it's not in the queue anymore. Let's schedule a job for later. Let's, uh, not perform later, but let's say, uh, set wait one week dot perform later. Now we'll go here. You see, we have, uh, one job that is scheduled to run in seven days. So, okay, salt Queue is running locally, but Salt Queue has another really great feature. It can plug it in directly into Puma. So if you open Puma rb, you have this, uh, environment Salt Queue in Puma. So if you enable it, then salt queue will run in the main, uh, process. And, uh, you don't need to create a separate, uh, worker for it. So I will, uh, uh, it, I will, uh, uh, disable the worker or remove the worker from the, uh, profile. Let's try removing it. And, uh, instead I will, uh, go to Puma and, uh, run. So queue if Rails environment is development. So again, I will start the rails, uh, server, just the rail server. I don't need to start the pro profile. I can even add this back to the pro profile. I'm just running the rail server and, uh, let me run another job Rails console. So hello Rail job perform later. Now we have two jobs that I finished. I will add this job and we have three jobs that I finished. So you see, I don't need to even, uh, add the, I can just, uh, say that, uh, I will run sold queue inside Puma in the development environment and in production. I would need to, uh, enable this environment. Sold queue in Puma equals through. So, uh, yes, it looks like we have, um, everything running locally. We have, uh, mission control installed. Have, uh, sold queue that focus locally inside the main web process. Now we need to deploy to production and we're going to deploy to production in this case to, uh, render. Uh, I like render. I am actually hosting cpr.com on, uh, render. And it also has, uh, accessibility to host a, uh, database and, uh, a web service for free. Nowadays, not many, uh, platforms as a service have this free option and render does have a free option. So we're going to try to host, uh, uh, money Gun for free. Uh, let's go and create a new, uh, web service. I will use, uh, Monon. You can also define the public GI repository. You can just go to, yeah, we slash monon. I'm going to paste it, uh, right here. Connect. Uh, now the language is going to be Ruby. Uh, the branch is not going to be main, it's going to be the branch where I did all the changes. So I will create a new branch, let's say, um, solid production render. Let's, uh, save all these changes. So we added the job. We removed HP basic authentication from mission control. Uh, set the primary database to be used in the sold cable sold cache. Uh, added our API keys for Stripe in the previous episode. Uh, removed the settings from, uh, the production database v ml to use the main database. Uh, in development. We're not using inline, we're using solid queue in production. We are also using solid queue and we're using a separate database. Uh, we are using, uh, inline, uh, solid queue in Puma in development environment. Uh, again, we are using the jobs, uh, dashboard. We previously set up the stripe prices. We added the migrations to the main database for solid. And, uh, yes, that's it. So let's save and let's go back here. Now it doesn't have this, uh, brunch that I've been playing with, so I will need to, uh, restart because, well, this branch wasn't present. Okay, let's see. The brunch, uh, what's the name of the brunch? Solid production. Oh yeah, the brunch isn't there yet because I just, just, uh, created it. Okay, so I, uh, refreshed and now have the solar potass around the branch. Uh, I'm going to use the language. Uh, Ruby, it is important. Now, rule directory doesn't matter. Bundle install here in end, I'm going to say bundle exec rails, DB migrate, run the migrations in the end. Um, here I can just, uh, say bundle, execute the rail server. I don't think I need all this stuff, but let's try going with it. Let's like the free option for now and we need to set up the Rails master key. So I will add the master key that we set up here. Then I will add the, the environment variable sold queue in Puma to True. And we'll also need to set up the database and we, the database, your Alpha this. And we need to separately create a database in render. So in the meantime, let's try to, uh, deploy the web service. I think it'll fail. Yeah. Here I will also set the health check part to slash op. Let's click on Deploy, but it'll fail because it don't have a database, uh, connected. In the meantime, I will go and create a new database. So new Postgres database, um, Monken Postgres. Uh, we don't need to set anything up, uh, except for this. Let's use a free database for now. Oh, okay. Create database. And, uh, when the database is created, we should be able to get an, uh, internal database. CRL here. Yeah, we already have this CRL available, but it'll not work yet because the database is still, uh, in the creation phase. We can, in the meantime, go and, uh, add this database, URL to the Money Gun Web Service, going into environment variables. Here I will go edit, uh, new variable database, URL. Um, let's just save for now without redeploying because, uh, it doesn't make sense until the database is, uh, fully created, legislate a bit. Okay, so the Postgre Scale database has been, uh, created and we have a failed deploy on the main repository. Let's just have look why, uh, yes, because, uh, we don't have PostgreSQL set up. So now we have it, uh, connect. We have the database URL set up, and now we can redeploy. So I will clear build, cache and deploy, and let's see if it works this dime. Okay, so now it says build successful. Let's go back to the dashboard. And you see, uh, Postgres is okay, I will refresh and, uh, money Gun seems to be running. Let's try to open it. And yes, it, uh, loads. Now let's try signing in or create a new account. Let's try to create an organization and let's try to subscribe. So get started. I think, uh, they're going to have a problem because we need to set up web hooks to work with the, the new created production URL with Money Gun on gender.com. So anyway, let's try going through. Let's, uh, pay. In the meantime, we can have a look at our dashboard. So this is the new created customer that we already created from production, not from our development environment. And, uh, let's see if the payment gets processed. For some reason, Stripe is, uh, running quite slow for me today. Okay. It's been processed, so I'm redirected back and, uh, going here, I will go to the customer here. He's subscribed. He has an active subscription, but, uh, here I don't see him having an active subscription. But if I go here, uh, to reinitialize the subscription, it'll make an API request to Stripe to check if he's subscribed and it'll say that he's subscribed. And, uh, I think we will redirect back showing that we have an active subscription. You see it as a free instance, so it might be the reason why it's running. So, uh, slow. Yeah, you see now we have an active subscription. So now let's set up Stripe Webhooks to deliver to the production. Uh, URLI will go to developers, uh, webhooks. And, uh, here are previous levels already setting up some kind of URL money gun on render.com. I think it's going to be more or less the same. URL mangan on render do com, pay Webhook, Stripe. And I will just need to use, uh, this, uh, webhook key, uh, and add it to my, uh, credentials. So I will go with this and I will add this, uh, webhook sign key to my credentials. So, uh, I need to add this, uh, webhook secret. This is the one that I was using for development, and this is the one for my production environment. But again, I'm using the, that test keys for both production and development at the moment. Let's, uh, save. Now actually, if you open the webhook endpoint, you will see I'm listening to 19 different events. And these are all the events that we need to listen to according to the page, GM documentation, and then trading and new webhook. You don't need to add them all manually. Like if I go here, click on add endpoint, I would be, uh, suggested to add all these events manually. Uh, instead there is better thing that they can do. If you go to the money gun repository and, uh, find the webhooks here to enable web hooks, you can click here for that development environment or here for production environment. So I will click here and, uh, it has, uh, all the web hooks encoded in the UL brands, so you don't need to do it manually. Here you see all of the 19 endpoints have been, uh, added, and all you need to do is add the production URL. So I'll go to dashboard render.com. What is the production? URL? Right now I have redeployed the application. So here it is. And, uh, I need to have slash pay. Uh, what was it? Let's see. In the docs, read me, uh, pay a web Hooks, stripe, Okay, at Endpoint. Now you can copy the silent secret, add it to your credentials, save. And, uh, I'm going to push these changes. And now whenever I'm pushing changes, uh, uh, Randy is going to redeploy my application. So, uh, let's have a look here. You see, it is deployed once again because, uh, I made some code changes if I go to events, you see, uh, just now when I pushed the latest change with the credentials, it is redeploying the application. Okay? So now we redeploy the application with the, the added web hook key. And now let's try, uh, seeing if the webhooks are delivered. So I will create a new organization. Again, I'm on the free plan, so it can, uh, take some time. Now going to billing, uh, subscribe. And the expectation when I subscribe is that the web hook will be delivered and the, the user will be subscribed right away. Here. Redirects back to the application. Yeah, processing the payment. Stripe is running all its magic. We can already maybe go back to our customers and see there is this new other customer. Let's have a look at our web hooks. So you see previously we had a bunch of web hooks that were not delivered. And, uh, the arrive now, okay, yeah, I'm back in this, our organization. And, uh, you see the subscription is active. So I was just iterate back and the subscription is active. And if I refresh the web hooks dashboard, You see, uh, all the web hooks that we have sent after and the web hook key have been delivered. Now, yes, we have, uh, money gone running on agenda. Everything is good. The only thing that is, uh, imperfect is that we are using the same Stripe, API keys for, uh, development. And for production we are using the test keys. And you would want to use the test keys for like, the test mode for development, maybe for station, but for production, you would want to use the production keys. And here we have everything set up in this credentials file just for, uh, one Stripe environment. So you would be able to add the, either like a separate, uh, folder for production and the development here, or you can, uh, go with the Rails credentials approach to have separate credentials for development. For test, for production, something like this. We can run real credentials, edit environment development Here we would be able to add our development keys. We would create a separate, uh, one for production, add production keys here, and uh, it would create the folders, so the folder for credentials, development and production. And you would just add the master key for production or master key for development, uh, in your environment. And this way you would be able to have different stripe keys for development and production. Okay, so that's it. This was quite a lot of work. We set up solid TriFactor to work with PostgreSQL, and we deployed the application to production. Thanks for watching and see you in the next.