Feature #4024
closed[ce] Integrations/Webhooks
0%
Description
Currently there is a WIP to implement integrations in the rhodecode ui, in a way that is simple for an end user to configure.
There are a few tasks which must be done to make this work:
Create a unified first class citizen event object system, these should be generated in relevant models eg. PullrequestModel.create to create and fire PullRequestCreateEvent(pullrequest) - they should also gather relevant environment data such as request context - each event has a string describing it, of the form 'repo.push' and a special wildcard
'*'
string will match any eventThese objects should expose a json dict which describe the entirety of the event - such as repo (plus repo data), user, relevant commits etc. - It would be a good idea to use the data structs returned by the rhodecode api, eg.
{'event': 'pullrequest.create', 'repo': repo.get_repo_api_data()}
This json dict is the basis of all integrations/webhooks - it must be as complete as possible to allow any integration/webhook to utilize the data without needing further apicalls/lookups (later we can allow post_processors "extensions" to be registered on the events - to let them add extra data).
Now that we have a basis to describe an event - we can work on hooks - A table of event hooks of the form:
CREATE TABLE event_hooks (
event_type VARCHAR,
integration_id INT
)
and an integrations table of the form:
CREATE TABLE integrations (
integration_id INT primary_key,
# this is a string which is used as a
# discriminator to load the correct integration
# eg. 'slack' => SlackIntegration
integration_type VARCHAR,
# this is a json string which exposes settings for the integration
# these will be exposed in the user interface as a form
integration_settings_json VARCHAR
# when this is null, will apply the integration on any repo
# instead of having to create a new integration per repo
# you can make all repo.push use same integration
# it will also be null for global type events such as `user.create`
repo_id INT,
)
which maps eg. events => integrations, where integration will be database objects that inherit from a base Integration class that define how to interact with the incoming json data - and do the analogous integration - for example with these values:
+------------+----------------+
| event_type | integration_id |
+------------+----------------+
| repo.push | 1 |
| repo.pull | 1 |
| repo.pull | 2 |
| * | 3 |
+------------+----------------+
+----------------+---------+------------------+---------------------------------+
| integration_id | repo_id | integration_type | integration_settings_json |
+----------------+---------+------------------+---------------------------------+
| 1 | 1 | slack | '{"channel": "#develop"}' |
| 2 | null | slack | '{"channel": "#general"}' |
| 3 | null | webhook | '{"url": "http://logs-server"}' |
+----------------+---------+------------------+---------------------------------+
A push to repo(id=1) will fire two integrations:
- integration(id=1) - sends a message to slack channel #develop
- integration(id=3) - POSTS the repo push json to http://logs-server
A pull to repo(id=1) will fire three integrations
- integration(id=1) - sends a message to slack channel #develop
- integration(id=2) - sends a message to slack channel #general
- integration(id=3) - POSTS the repo push json to http://logs-server
A pull to repo(id=2) will fire two integrations
- integration(id=2) - sends a message to slack channel #general
- integration(id=3) - POSTS the repo push json to http://logs-server
In fact any event will be posted to the http://log-server - and any pull event will be sent to slack #general.
Updated by Daniel D over 8 years ago
The Integration base class should follow a pyramid class structure similar to auth plugins that makes it auto expose in the admin making new integrations easy to add
Updated by Marcin Kuzminski [CTO] over 8 years ago
few notes:
- could wildcard work like glob patterns, so one could use
repo:*
would be cool if you can define an events for repo group as well. When we did per-repo settings we prepared some work for to make it also work for per-repo groups. However i'm not sure if this could be fast enough to determine an event for a member of groups (they could be nested indefinetly)
i'm not sure if we should use
repo.get_repo_api_data()
maybe more explicit function likerepo.get_event_data()
would be better even if at beginning callsget_api_data()
this would allow us to have some flexibility on how stable the events data is.
I think we also should consider detection of same events, what if there's a global event, and you define same action on per-repo basis, or using other patterns. I think it's easy to check simply by hashing event type and it's params so same hashes can be skipped.
Any thoughts ?
Updated by Daniel D over 8 years ago
Yep glob patters would work for the nested event type - the only issue I found there was the scheme would need some design eg:
repo
push
pre
post
pull
pre
post
pullrequest
create
merge
update
+1 on get_events_data
Hashing method to check same event + integration would require a stable settings json - something to keep in mind..maybe sort it before saving.
Updated by Marcin Kuzminski [CTO] over 8 years ago
- Target version set to v4.2
- Private changed from Yes to No
Updated by Marcin Kuzminski [CTO] over 8 years ago
- Target version changed from v4.2 to v4.3
Updated by Marcin Kuzminski [CTO] over 8 years ago
- Status changed from New to In Progress
Updated by Daniel D over 8 years ago
- Status changed from In Progress to Feedback
Events/Integrations have been added to 4.3, we're now in testing phase.
Updated by Marcin Kuzminski [CTO] over 8 years ago
- Status changed from Feedback to Closed