Like many other Nova objects, server groups (which can be used to provide affinity/anti-affinity scheduling policy) are not supported by Nova cells.

The core problem is the server groups created by nova-api only get populated in the API cell database.  They aren’t put into the compute cell database(s), and thus aren’t available to nova-scheduler.  Instance flavors have the same problem.

Getting Server Groups into the Compute Cell DB

Typically OpenStack operators using nova-cells will have to manually populate this information to the compute cell database(s).  This is a workable solution for flavors, since those are normally quite static.

This manual method doesn’t work for server groups, since those can be created/modified/deleted by end users at any time.

The information flow for server groups data (and flavor data) is always one way: API cell to compute cell.  Because of this, we can automatically populate these tables in the compute cell from the API cell using SQL triggers.

We could also use this same SQL trigger method for populating flavors info into the compute cell DB.

(See also: http://www.whycloudcomputing.net/2012/08/mysql-trigger-different-databases-examples/ for general case triggers across databases.)

Caveats

There are always gottchas:

  • This only works if the API cell and compute cell databases are running in the same MySQL instance.
  • The database user which is used to access the API cell Nova database must also have CRUD access to the compute cell Nova database.
  • You need a set of triggers for each compute cell.  Obviously this doesn’t scale well for more than a small number of cells.
  • I am not familiar with the internals of MySQL and it’s implementation of triggers.  I don’t believe this should significantly increase database load, but that could be wrong.
  • I don’t think this would interfere with future nova DB schema migrations.  But by having triggers we implicitly assume that the API cell and compute cell databases are always at the same schema version.
  • Obviously this is just a work around until Cells v2 is ready in Liberty.  Doing this under the covers and without Nova’s explicit knowledge may have unintended consequences.  Caveat emptor!

SQL Triggers for Server Groups

Here are the triggers that need to be created on the API cell database.

Note that there is a CREATE, UPDATE, and DELETE trigger for each of the instance_groups, instance_group_policy, and instance_group_member tables (a total of 9 triggers for each compute cell database.)

DELIMITER &&

create trigger after_insert_nova_api_cell_instance_groups after insert on `nova_api_cell`.instance_groups for each row begin insert into `nova`.instance_groups(created_at, updated_at, deleted_at, deleted, id, user_id, project_id, uuid, name) values (NEW.created_at, NEW.updated_at, NEW.deleted_at, NEW.deleted, NEW.id, NEW.user_id, NEW.project_id, NEW.uuid, NEW.name); END &&

create trigger after_insert_nova_api_cell_instance_group_policy after insert on `nova_api_cell`.instance_group_policy for each row begin insert into `nova`.instance_group_policy(created_at, updated_at, deleted_at, deleted, id, policy, group_id) values (NEW.created_at, NEW.updated_at, NEW.deleted_at, NEW.deleted, NEW.id, NEW.policy, NEW.group_id); END &&

create trigger after_insert_nova_api_cell_instance_group_member after insert on `nova_api_cell`.instance_group_member for each row begin insert into `nova`.instance_group_member(created_at, updated_at, deleted_at, deleted, id, instance_id, group_id) values (NEW.created_at, NEW.updated_at, NEW.deleted_at, NEW.deleted, NEW.id, NEW.instance_id, NEW.group_id); END &&

create trigger after_update_nova_api_cell_instance_groups after update on `nova_api_cell`.instance_groups for each row begin update `nova`.instance_groups set created_at = NEW.created_at, updated_at = NEW.updated_at, deleted_at = NEW.deleted_at, deleted = NEW.deleted, id = NEW.id, user_id = NEW.user_id, project_id = NEW.project_id, uuid = NEW.uuid, name = NEW.name where deleted = OLD.deleted and id = OLD.id and user_id = OLD.user_id and project_id = OLD.project_id and uuid = OLD.uuid and name = OLD.name limit 1; END &&

create trigger after_update_nova_api_cell_instance_group_policy after update on `nova_api_cell`.instance_group_policy for each row begin update `nova`.instance_group_policy set created_at = NEW.created_at, updated_at = NEW.updated_at, deleted_at = NEW.deleted_at, deleted = NEW.deleted, id = NEW.id, policy = NEW.policy, group_id = NEW.group_id where deleted = OLD.deleted and id = OLD.id and policy = OLD.policy and group_id = OLD.group_id limit 1; END &&

create trigger after_update_nova_api_cell_instance_group_member after update on `nova_api_cell`.instance_group_member for each row begin update `nova`.instance_group_member set created_at = NEW.created_at, updated_at = NEW.updated_at, deleted_at = NEW.deleted_at, deleted = NEW.deleted, id = NEW.id, instance_id = NEW.instance_id, group_id = NEW.group_id where deleted = OLD.deleted and id = OLD.id and instance_id = OLD.instance_id and group_id = OLD.group_id limit 1; END &&

create trigger after_delete_nova_api_cell_instance_groups after delete on `nova_api_cell`.instance_groups for each row begin delete from `nova`.instance_groups where deleted = OLD.deleted and id = OLD.id and user_id = OLD.user_id and project_id = OLD.project_id and uuid = OLD.uuid and name = OLD.name limit 1; END &&

create trigger after_delete_nova_api_cell_instance_group_policy after delete on `nova_api_cell`.instance_group_policy for each row begin delete from `nova`.instance_group_policy where deleted = OLD.deleted and id = OLD.id and policy = OLD.policy and group_id = OLD.group_id limit 1; END &&

create trigger after_delete_nova_api_cell_instance_group_member after delete on `nova_api_cell`.instance_group_member for each row begin delete from `nova`.instance_group_member where deleted = OLD.deleted and id = OLD.id and instance_id = OLD.instance_id and group_id = OLD.group_id limit 1; END &&

DELIMITER ;

Triggers for Flavors

We have not actually implemented this scheme for instance flavors, however the setup should pretty much be the same.  You’d need triggers for the instance_typesinstance_type_projects, and instance_type_extra_specs tables, I believe.

Quotas for Server Groups

Be aware that there are server-groups and server-group-members quota limits, which both default to 10.  If you want your users to be able to use more than 10 server groups and/or have more than 10 servers in each group, you will need to update these quota limits.