How to deploy PHP/HTML website with Gitlab CI

Setting up Gitlab-CI to deploy your PHP & HTML (or any other ‘static’ site) is quite simple, but poorly documented. After a few weeks of fiddling around I found the following solution. I’d love to hear everyone’s feedback.

Requirements:

  1. Your webserver must have root access (to install the runner).
  2. Your site/codebase should be rather small and not too complex.
    • This example uses rsync to move/sync two folders, so sites that can self-update their DB like wordpress should be OK, but I have not tested it.
    • We’re going to use rsync to move/sync two folders, so sites that self-manage media content, like wordpress, will lose that media content. You will need to tweak the rsync script to handle this yourself.
  3. A gitlab install or repo you have permission to add runners to.
  4. These instructions are for Linux (Ubuntu 14.04 specifically) but they should be applicable to any OS supported by the runner.

Step 1 – Install the Runner

Install the Gitlab Runner on your web server.

I’m not going to go over this one in much detail (none actually) as the documentation for getting the runner installed is pretty good.

Step 2 – Setup the Runner

If you’re like me and you run your own private Gitlab install, but are not *that* familair with git and git tagging you might become confused about how the runner tagging works, since you can assign tags to entire repos in gitlab.

gitlab_runner_settings

The Gitlab Runner settings screen

On the Runner settings, found in the admin section under Runners, then clicking on the runner token number, you have an option to set which tags will fire this runner. This corresponds to the tags on individual commits, and not the tag set to the repo. If you want specific repo’s to always use specific runners, such if you have different webservers, you will need to manually assign the runner to that repo. You can do that from the runner’s setting screen, or from the repo. Unless you want to the madness that is Git Tagging I suggest you leave the “Tags” field empty and the “Run untagged jobs” box checked.

Step 3 – Configure the Repo

3.1 – Enable CI Builds

gitlab_project_features

Make sure “Builds” is checked.

Go to your repo settings. In Gitlab 8.11 that can be done by clicking the gear icon in the upper right corner of the screen when looking at any other “Project”  page .

Gear -> Edit Project

Scroll down until you see “Features” and make sure that “Builds” is checked.

 

In this case I don’t need any of the other features so I uncheck them. You don’t have to, but if you aren’t going to use those features it will clean up the Project overview screen a good bit if you turn them off.

3.2 – Set Project Variables

gitlab_project_gear_dropdown_variables

The options you see here will vary depending on what features you have enabled, and other settings in your project.

This one is sort of optional, but if you have multiple small projects you can use variables in your build script (you’ll see it in a couple of steps) that allows you to re-use the same build script in multiple projects and set the few unique variables from the project itself.

You get to the variables settings page from any other page in the project by clicking on the Gear icon in the upper right.

Gear -> Variables.

Set the Key to “webroot” and the value to the web path on your webserver where your site’s files reside. For example, it’s normally /var/www/, so I set “/var/www/mydomain.com/”. The trailing slash is important here.

gitlab_project_variables

You should see this once you’ve added the new variable.

Step 4- Add build Script

You might see some errors around the project indicating that builds are not configured or working correctly. The specific message seems to be to change from one Gitlab version to another, and from one screen to another, but all of them are indicating that you have builds enabled for your project, but no build script.

Create a file named “.gitlab-ci.yml” in the root of your project. Note the leading period is important. If you’re trying this from Windows it might give you grief about it, I ended up kidnapping a .htaccess file from another project and renaming it.

Open the file in your preferred text editor and set this as the contents:

stages:
 - deploy

pages:
 stage: deploy
 script:
 - if [ -z "$webroot" ]; <yoastmark class='yoast-text-mark'>then echo "Need to set webroot" &</yoastmark>amp;amp;amp;amp;& exit 1; fi
 - sudo rsync -rv ./ $webroot --exclude '.git' --exclude '.gitlab-ci.yml' --exclude '.gitignore' --delete
 - sudo chown user:group $webroot -R
 only:
 - master

NOTE: WordPress likes to strip leading spaces. Every line except “stages:” and “pages:” are indented 2 spaces.

So whats going on here? This is a Gitlab CI build script, and it basically instructs the CI to do various tasks. Normally this is used to compile code, check it for simple mistakes, etc. We’re going to re-purpose its ability to execute bash/batch commands (even on windows) to push our files where we want.

You can read the full documentation here. I’ll break down my script for you if your eyes got a little glossed over after trying to ingest the official help documents.

  • Line 1 tells CI that we’re declaring our stages. You can have as many stages as you want, but they must all be either “build”, “test” or “deploy”. Since we’re not compiling or testing code, we just declare a deploy on Line 2.
  • Line 4 is your stage name. You are free to call this whatever you want. Since I was working off the official “Gitlab Pages” build instructions, my build is called “pages”.
  • Line 5 declares that this is our deployment stage.
  • Line 6 declares that the following lines will be our batch script.
  • Lines 7-9 are batch scripts, executed sequentially. Note that on linux systems these should be bash scripts, while on Windows you should use batch syntax and commands.
    • Line 7 is a sanity check for our variable. If it isn’t set, this check dies and returned a 1 exit code, which causes the entire build to stop and fail. This prevents rsync from doing anything funny if you forgot to set the variable, and if you have build notifications enabled, you’ll be prompted with the text “Need to set webroot”.
    • Line 8 is the real meat & potatos of this whole thing. Since this commands run from the temp folder the runner makes for itself, this grabs the entire contents of the current folder “./” and rsyncs it to our webroot folder. You did set that variable, right?
      • The three exclude flags tell rsync to ignore the git & gitlab specific files. You’re free to add more if your project needs it.
      • The –delete flag tells rsync to delete anything in the webroot folder that isn’t in the new temp folder. If you delete a file from your repo, this step makes sure its also cleared from your webroot.
      • If you like to keep your web-facing code someone else other than the root of your repo (such as /pub_html or some such) you will need to adust the “./” to be “./pub_html” or w/e you use.
    • Line 9 changes the ownership of the new files in the webroot to be whatever owner & group you want. This isn’t strictly needed, if your project is strictly read-only from the web server side of things this step can be skipped. I like to have this set to same owner & group as if I had uploaded the files, in the event I need to SSH in and make a quick change to (zomg, testing in prod?) something real quick to appease the Powers-That-Be before committing the change to the repo.
    • Lines 10 & 11 makes it so that this job will only fire for changes to the ‘master’ branch. You could use this to have multiple branches deploy to different folders if you wanted, but you would need to extend the variables and/or adjust the rest of the script.

If you want to keep the ‘sudo chown’ command in there, you’ll need to make a sudoers change to allow it. On Ubuntu & debian:

sudo visudo

add at the bottom:

gitlab-runner ALL=(ALL) NOPASSWD: ALL

This will allow the gitlab runner to run any sudo command. It is a potential security issue being wide open like that, and if it concerns you you can lock it down a bit tighter.

How to setup Let’s Encrypt for Gitlab

This is a quick guide to setting up Let’s Encrypt for Gitlab

Assumptions made:

  1. You’re using the Omnibus Gitlab install.
  2. You’re using Ubuntu (or Debian).
  3. Gitlab is available on the root of a subdomain (gitlab.yourdomain.com)

Read more »

Howto: Resolve “Failed to Open File” error when installing Magneto Connect extensions

If you’re trying to install a Magento extension though Magento Connect and getting an error like the following:

CONNECT ERROR: Failed to open file /var/www/magento/downloader/.cache/community/OrganicInternet_SimpleConfigurableProducts-0.7.4/app/code/community/OrganicInternet/SimpleConfigurableProducts/Catalog/Model/Product/Type/Configurabl

It’s caused by a simple (yet long-lived and un-fixed) bug in Magento’s downloader.

Open:

/downloader\lib\Mage\Archive\Tar.php

Find:

if (!($header['name'] == [email protected]' && $header['type'] == 'L')) {
     $header['name'] = trim($header['name']);
     return $header;
}

Replace with:

if (!(trim($header['name']) == [email protected]' && $header['type'] == 'L')) {
     $header['name'] = trim($header['name']);
     return $header;
}

Then try to install your extension again. Always remember to disable the compiler before hand, clear caches, and log out of the admin after installing an extension.

HowTo: Remove message requirement when assigning tickets in OSTicket

Sometimes you just need to assign a ticket to another team member quickly and move on, but OSTicket insists on a leaving an assignment message. This is especially annoying when the reason for the assignment is blatantly obvious (such as a sales inquiry being assigned to a sales rep in the dept).

In OSTicket v1.9.7 open scp/tickets.php and locate the following block (around line 171):

 //Comments are not required on self-assignment (claim)
if($claim && !$_POST['assign_comments'])
$_POST['assign_comments'] = sprintf(__('Ticket claimed by %s'),$thisstaff->getName());
elseif(!$_POST['assign_comments'])
$errors['assign_comments'] = __('Assignment comments required');
elseif(strlen($_POST['assign_comments'])<5)
$errors['assign_comments'] = __('Comment too short');

If you want to remove the comment requirement altogether then either comment out or remove the last 4 lines like so:

//Comments are not required on self-assignment (claim)
if($claim && !$_POST['assign_comments'])
$_POST['assign_comments'] = sprintf(__('Ticket claimed by %s'),$thisstaff->getName());

If you just want to shorten the minimum requirement, then change the 2nd to last line.

From:

elseif(strlen($_POST['assign_comments'])<5)

to:

elseif(strlen($_POST['assign_comments'])<3)

A quick guide to Solving: Error code: ssl_error_rx_record_too_long

If you’re getting the error “Error code: ssl_error_rx_record_too_long” when trying to setup an SSL enabled website on Apache (2.4 specifically, but likely 2.2 as well), double check that you’ve actually enabled SSL in the virtualhost config file. Its very easy to get carried away imputing all the various SSL parameters and forget the most important:

SSLEngine on

Add customers to Magento newsletter via web form form

Recently I was tasked with adding customers to a Magento newsletter via a pre-exisitng form. Previously this form just sent an email that was mostly ignored, so now the ManInCharge wanted it feeding into Magento, which has several nice plugins for MailChimp integration, but that’s a story for another day.

Setup your form just like any other HTML text form, point it with a POST action at something like this:

$email = $_POST["email"];
if ($email){

# create new subscriber without send an confirmation email
Mage::getModel('newsletter/subscriber')->setImportMode(true)->subscribe($email);

# get just generated subscriber
$subscriber = Mage::getModel('newsletter/subscriber')-&gt;loadByEmail($email);

# change status to "subscribed" and save
$subscriber->setStatus(Mage_Newsletter_Model_Subscriber::STATUS_SUBSCRIBED);
$subscriber->save();
}

With some help from ifuelinteractive and paniticz

Monitor S.M.A.R.T. stats in Zabbix

Need to track disk SMART stats in Zabbix? I found a fairly simple method that does not rely on external scripts (other than the Zabbix agent).

1) Edit your Zabbix Agent config to permit remote commands if you have not already done so. It’s usually /etc/zabbix/zabbix_agentd.conf

EnableRemoteCommands=1

2) Near the bottom of your agent config there should be several “UserParamerter=…” lines, add a new one:

UserParameter=hdd.smart[*],sudo smartctl -A /dev/$1 | grep -E -i '^[ ]*($2)[ ]' | cut -c88-

In short, this command spits out a full SmartMonTools report for your drive ($1), greps it for a single specific line ($2), then removes the first 88 characters, leaving only the raw value behind.

Make sure that smartctl is in your suroers file for any user to run without a password prompt. I detail that process in a previous post.

Read more »

Allow any user on linux to run smartctl without password

Need to have a script or external application run smartctl without being prompted for a password? Simply add it to the sudoers file. Under Ubuntu/Debian use “visudo” to edit it (DO NOT EDIT IT WITHOUT USING THIS COMMAND!) and add the following line:

ALL ALL=(ALL)NOPASSWD: /usr/sbin/smartctl

This allows any user, from any source (local or remote) to run the smartctl command without being prompted for a password. Note, your script or user will still need to preface the smartctl command with sudo.