Feature #4024
[ce] Integrations/Webhooks
0%
Description
Currently there is a WIP to implement integrations in the rhodecode ui, in a way that is simple for an end user to configure.
There are a few tasks which must be done to make this work:
Create a unified first class citizen event object system, these should be generated in relevant models eg. PullrequestModel.create to create and fire PullRequestCreateEvent(pullrequest) - they should also gather relevant environment data such as request context - each event has a string describing it, of the form 'repo.push' and a special wildcard
'*'
string will match any eventThese objects should expose a json dict which describe the entirety of the event - such as repo (plus repo data), user, relevant commits etc. - It would be a good idea to use the data structs returned by the rhodecode api, eg.
{'event': 'pullrequest.create', 'repo': repo.get_repo_api_data()}
This json dict is the basis of all integrations/webhooks - it must be as complete as possible to allow any integration/webhook to utilize the data without needing further apicalls/lookups (later we can allow post_processors "extensions" to be registered on the events - to let them add extra data).
Now that we have a basis to describe an event - we can work on hooks - A table of event hooks of the form:
CREATE TABLE event_hooks ( event_type VARCHAR, integration_id INT )
and an integrations table of the form:
CREATE TABLE integrations ( integration_id INT primary_key, # this is a string which is used as a # discriminator to load the correct integration # eg. 'slack' => SlackIntegration integration_type VARCHAR, # this is a json string which exposes settings for the integration # these will be exposed in the user interface as a form integration_settings_json VARCHAR # when this is null, will apply the integration on any repo # instead of having to create a new integration per repo # you can make all repo.push use same integration # it will also be null for global type events such as `user.create` repo_id INT, )
which maps eg. events => integrations, where integration will be database objects that inherit from a base Integration class that define how to interact with the incoming json data - and do the analogous integration - for example with these values:
+------------+----------------+ | event_type | integration_id | +------------+----------------+ | repo.push | 1 | | repo.pull | 1 | | repo.pull | 2 | | * | 3 | +------------+----------------+ +----------------+---------+------------------+---------------------------------+ | integration_id | repo_id | integration_type | integration_settings_json | +----------------+---------+------------------+---------------------------------+ | 1 | 1 | slack | '{"channel": "#develop"}' | | 2 | null | slack | '{"channel": "#general"}' | | 3 | null | webhook | '{"url": "http://logs-server"}' | +----------------+---------+------------------+---------------------------------+
A push to repo(id=1) will fire two integrations:
- integration(id=1) - sends a message to slack channel #develop
- integration(id=3) - POSTS the repo push json to http://logs-server
A pull to repo(id=1) will fire three integrations
- integration(id=1) - sends a message to slack channel #develop
- integration(id=2) - sends a message to slack channel #general
- integration(id=3) - POSTS the repo push json to http://logs-server
A pull to repo(id=2) will fire two integrations
- integration(id=2) - sends a message to slack channel #general
- integration(id=3) - POSTS the repo push json to http://logs-server
In fact any event will be posted to the http://log-server - and any pull event will be sent to slack #general.
Updated by Daniel D over 4 years ago
The Integration base class should follow a pyramid class structure similar to auth plugins that makes it auto expose in the admin making new integrations easy to add
Updated by Marcin Kuzminski [CTO] over 4 years ago
few notes:
- could wildcard work like glob patterns, so one could use
repo:*
would be cool if you can define an events for repo group as well. When we did per-repo settings we prepared some work for to make it also work for per-repo groups. However i'm not sure if this could be fast enough to determine an event for a member of groups (they could be nested indefinetly)
i'm not sure if we should use
repo.get_repo_api_data()
maybe more explicit function likerepo.get_event_data()
would be better even if at beginning callsget_api_data()
this would allow us to have some flexibility on how stable the events data is.
I think we also should consider detection of same events, what if there's a global event, and you define same action on per-repo basis, or using other patterns. I think it's easy to check simply by hashing event type and it's params so same hashes can be skipped.
Any thoughts ?
Updated by Daniel D over 4 years ago
Yep glob patters would work for the nested event type - the only issue I found there was the scheme would need some design eg:
repo push pre post pull pre post pullrequest create merge update
+1 on get_events_data
Hashing method to check same event + integration would require a stable settings json - something to keep in mind..maybe sort it before saving.
Updated by Marcin Kuzminski [CTO] over 4 years ago
- Target version set to v4.2
- Private changed from Yes to No
Updated by Daniel D over 4 years ago
- Status changed from In Progress to Feedback
Events/Integrations have been added to 4.3, we're now in testing phase.