diff --git a/README.md b/README.md index af40be8..5b24d2e 100644 --- a/README.md +++ b/README.md @@ -1,314 +1,314 @@ # Sylk Pushserver [Home page](http://sylkserver.com) Copyright (C) 2020 AG Projects Sylk Pushserver was designed to act as a central dispatcher for mobile push notifications inside RTC provider infrastructures. Both the provider and the mobile application customer, in the case of a shared infrastructure, can easily audit problems related to the processing of push notifications. Author: Bibiana Rivadeneira ## License Sylk Pushserver is licensed under GNU General Public License version 3. [Copy of the license](http://www.fsf.org/licensing/licenses/gpl-3.0.html) ## Deployment scenarios Sylk Pushserver can be deployed together with WebRTC server applications or VoIP servers like SIP Proxies and PBXs. Its main purpose is to act as a central entity inside RTC provider infrastructures. Without such a component, the same functionality must be built inside multiple servers and as the number of mobile applications increases, the need for such central component becomes obvious. ### Integration examples * OpenSIPS: **config/opensips.cfg** * SylkServer: built-in support ## Design Sylk Pushserver can handle an arbitrary number of different combinations of push notification service and mobile applications. It and can be extended by using Python programming language to support new push notification services and applications. Sample applications are provided to handle Sylk and Linphone mobile applications for Apple and Firebase push notification services. For each configured Apple application, the server maintains a persistent connection by using HTTP/2 over TLS 1.2 and reuses that connection for sending notifications related to the application. Latest voip functionality for iOS 13 or later is also suported. Each outgoing connection can use its own set of credentials, X.509 certificates and urls. The connection failures are properly handled and incoming requests remained queued for later by using a timer dependent on the payload type. ### Logging All incoming and outgoing requests, including HTTP headers and bodies, can be logged for troubleshooting purposes in the system journal and in a separate log file. These logs can easily be correlated with the logs from the server that generated the request by using the call-id key. Remote HTTP logging of the results is possible so that one or more third-parties can receive information about the individual push requests and responses for each application. ## API Sylk Pushserver expects a json over HTTP POST requests and translates it into a correspondent outgoing push notifications request to Apple Push Notifications or Firebase FCM servers. Json object structure: ```{ 'app-id': 'com.agprojects.sylk-ios', 'platform': 'apple', 'token': '6688-71a883fe', 'device-id': 'accc8375125582aae062353', 'call-id': '4dbe8-7a53-42bd-95f3-9a7d43938', 'from': 'alice@example.com', 'from_display_name': 'Alice', 'to': 'bob@biloxi.com', 'media-type':'audio', 'event': 'incoming_session' 'silent': True } ``` Where: * `app-id: str`, id provided by the mobile application (e.g. mobile bundle ID) * `platform: str`, 'firebase', 'android', 'apple' or 'ios' * `token: str`, destination device token, * *iOS device tokens* are strings with 64 hexadecimal symbols * *Android device push tokens* can differ in length`. * `device-id: str`, the device that generated the token * `call-id: str`, the unique session id for each call * `from: str`, address of the caller * `from_display_name`, (mandatory)*, display name of the caller * `to`, address of the callee * `media-type: str`: 'audio', 'video', 'chat', 'sms' or 'file-transfer' * `silent: bool`: *(optional, default `True`)* True for silent notification * `event: str`, type of event: * For *Sylk app* must be 'incoming_session', 'incoming_conference' or 'cancel' * For *Linphone app* must be 'incoming_session' The response is a json with the following structure: ```{ 'code': 'a numeric code equal to the HTTP response code', 'description': 'a detailed text description', 'data' : {} } ``` *data* contains an arbitrary dictionary with a structure depending on the request type and the remote server response. ### Sample client code See *scripts/sylk-pushclient* ### External APIs For documentation related to the API used by Apple and Firebase push notifications services you must consult their respective websites. For reference, the following APIs were used for developing the server, but these links may change: * [Sending Apple VoIp notifications](https://developer.apple.com/documentation/usernotifications/setting_up_a_remote_notification_server/sending_notification_requests_to_apns) * [Firebase Cloud Messaging](https://firebase.google.com/docs/cloud-messaging) - + * [FCM migration from legacy HTTP to HTTP v1](https://firebase.google.com/docs/cloud-messaging/migrate-v1) ## Installation ### As a Debian package Install the AG Projects debian software signing key: wget http://download.ag-projects.com/agp-debian-gpg.key sudo apt-key add agp-debian-gpg.key Add these repository matching your distribution to /etc/apt/sources.list: http://projects.ag-projects.com/projects/documentation/wiki/Repositories Update the list of available packages: sudo apt-get update sudo apt-get install sylk-pushserver ### From source The source code is managed using darcs version control tool. The darcs repository can be fetched with: darcs clone http://devel.ag-projects.com/repositories/sylk-pushserver Alternatively, one can download a tar archive from: http://download.ag-projects.com/SylkPushserver/ Install Python dependencies: `pip3 install -r requirements.txt` `python3 setup.py install` ### Building Debian package Install building dependencies: ``` sudo apt install dh-virtualenv debhelper libsystemd-dev dh-python python3-dev python3-setuptools python3-pip ``` Build the package: ``` python setup.py sdist cd dist tar zxvf *.tar.gz cd sylk_pushserver-?.?.? debuild ``` To install the debian package manually: ``` sudo dpkg -i sylk-pushserver_1.0.0_all.deb sudo apt --fix-broken install ``` ## Configuration There are two configurations files. * general.ini Contains the general server settings. * applications.ini Contains the settings for each mobile application, *see config/applications.ini.sample*. Chages to this file cause the server to autamtically reload it, there is no need to restart the server. ## Remote logging Remote logging is done using a POST request over HTTP with a json containg both the original request and the final response of the push notification. ```{ 'request': push_request, 'response': push_response } ``` Where : * push_request is the original json payload received by this server * push_response is a json with the following format: ```{ 'code': code, # http response code from PNS 'description': description, # detail description of the response from the PNS 'push_url': push_url, # the final URL of the outgoing push notification 'incoming_body': {...}, # the original request body received by the server 'outgoing_headers': {...}. # the outgoing request headers sent to the PNS 'outgoing_body': {...} # the outgoing request body sent to the PNS } ``` The returned result should be a json with a consistent key. The key can be defined in the application.ini for each application. If the key is set then its value will be logged which can make troubleshooting easier. ## Custom applications Custom applications can be written in Python by subclassing existing template classes. Define the directory for custom applications in `general.ini` file: `extra_applications_dir` = `/etc/sylk-pushserver/applications` Copy config/applications/myapp.py to the extra_applications_dir and overwrite its functions. In `applications.ini` file set app_type for the custom applications: ``` `app_type` = *myapp* ``` ## Custom Push services Custom PNS can be written in Python by subclassing existing template classes. Define the directory for custom push services in `general.ini` file: `extra_pns_dir` = `/etc/sylk-pushserver/pns` Copy config/pns/mypns.py to the extra_pns dir and overwrite its classes. In `applications.ini` file set app_type for the custom applications: ``` `app_platform` = *mypns* ``` ## Running the server ### From the source code `./sylk-pushserver --config_dir ` If the config_dir directory is not specified, the following paths are searched for: * /etc/sylk-pushserver *./config For more command line options use -h. ### Debian package ``` sudo systemctl start sylk-pushserver ``` ### Testing For testing the server scripts/sylk-pushclient can be used. ## Compatibility The server is developed in Python 3 and was tested on Debian Buster 10. ## Reporting bugs You may report bugs to [SIP Beyond VoIP mailing list](http://lists.ag-projects.com/pipermail/sipbeyondvoip/) diff --git a/pushserver/pns/firebase.py b/pushserver/pns/firebase.py index 0f21d37..c8c4fa4 100644 --- a/pushserver/pns/firebase.py +++ b/pushserver/pns/firebase.py @@ -1,350 +1,346 @@ import json import os import time from datetime import datetime import oauth2client import requests from oauth2client.service_account import ServiceAccountCredentials from pushserver.models.requests import WakeUpRequest from requests.adapters import HTTPAdapter from urllib3 import Retry from pushserver.pns.base import PNS, PushRequest, PlatformRegister from pushserver.resources.utils import log_event, fix_non_serializable_types class FirebasePNS(PNS): """ A Firebase Push Notification service """ def __init__(self, app_id: str, app_name: str, url_push: str, voip: bool, auth_key: str = None, auth_file: str = None): """ :param app_id `str`: Application ID. :param url_push `str`: URI to push a notification. :param voip `bool`: required for apple, `True` for voip push notification type. :param auth_key `str`: A Firebase credential for push notifications. :param auth_file `str`: A Firebase credential for push notifications. """ self.app_id = app_id self.app_name = app_name self.url_push = url_push self.voip = voip self.auth_key = auth_key self.auth_file = auth_file self.error = '' self.last_update_token = None self.access_token = self.set_access_token() def set_access_token(self) -> str: """ Retrieve a valid access token that can be used to authorize requests. :return: `str` Access token. """ + + #https://github.com/firebase/quickstart-python/blob/909f39e77395cb0682108184ba565150caa68a31/messaging/messaging.py#L25-L33 - if not self.auth_file: - self.error = f"Cannot generated Firebase access token, " \ - f"no auth file provided" - return '' - - if not os.path.exists(self.auth_file): - self.error = f"Cannot generated Firebase access token, " \ + if not self.auth_file or not os.path.exists(self.auth_file): + self.error = f"Cannot generate Firebase access token, " \ f"auth file {self.auth_file} not found" return '' scopes = ['https://www.googleapis.com/auth/firebase.messaging'] try: - credentials = ServiceAccountCredentials.from_json_keyfile_name(self.auth_file, - scopes) + credentials = ServiceAccountCredentials.from_json_keyfile_name(self.auth_file, scopes) oauth2client.client.logger.setLevel('CRITICAL') access_token_info = credentials.get_access_token() self.last_update_token = datetime.now() return access_token_info.access_token except Exception as e: self.error = f"Error: cannot generated Firebase access token: {e}" return '' class FirebaseRegister(PlatformRegister): def __init__(self, app_id: str, app_name: str, voip: bool, config_dict: dict, credentials_path: str, loggers: dict): self.app_id = app_id self.app_name = app_name self.voip = voip self.credentials_path = credentials_path self.config_dict = config_dict self.loggers = loggers self.error = '' self.auth_key, self.auth_file = self.set_auths() @property def url_push(self): try: return self.config_dict['firebase_push_url'] except KeyError: self.error = 'firebase_push_url not found in applications.ini' return None def set_auths(self): auth_key = None auth_file = None try: auth_key = self.config_dict['firebase_authorization_key'] except KeyError: try: auth_file = self.config_dict['firebase_authorization_file'] if self.credentials_path: auth_file = f"{self.credentials_path}/" \ f"{auth_file}" else: pass if not os.path.exists(auth_file): self.error = f'{auth_file} - no such file' except KeyError: self.error = 'not firebase_authorization_key or ' \ 'firebase_authorization_file found in applications.ini' return auth_key, auth_file @property def pns(self) -> FirebasePNS: pns = None if self.auth_key: auth_file = '' pns = FirebasePNS(app_id=self.app_id, app_name=self.app_name, url_push=self.url_push, voip=self.voip, auth_key=self.auth_key, auth_file=auth_file) elif self.auth_file: pns = FirebasePNS(app_id=self.app_id, app_name=self.app_name, url_push=self.url_push, voip=self.voip, auth_file=self.auth_file) self.error = pns.error if pns.error else '' return pns @property def register_entries(self): if self.error: return {} return {'pns': self.pns, 'access_token': self.pns.access_token, 'auth_key': self.auth_key, 'auth_file': self.auth_file} class FirebasePushRequest(PushRequest): """ Firebase push notification request """ def __init__(self, error: str, app_name: str, app_id: str, request_id: str, headers: str, payload: dict, loggers: dict, log_remote: dict, wp_request: WakeUpRequest, register: dict): """ :param error: `str` :param app_name: `str` 'linphone' or 'payload' :param app_id: `str` bundle id :param headers: `AppleHeaders` Apple push notification headers :param payload: `ApplePayload` Apple push notification payload :param wp_request: `WakeUpRequest` :param loggers: `dict` global logging instances to write messages (params.loggers) """ self.error = error self.app_name = app_name self.app_id = app_id self.platform = 'firebase' self.request_id = request_id self.headers = headers self.payload = payload self.token = wp_request.token self.wp_request = wp_request self.loggers = loggers self.log_remote = log_remote self.pns = register['pns'] self.path = self.pns.url_push self.results = self.send_notification() def requests_retry_session(self, counter=0): """ Define parameters to retry a push notification according to media_type. :param counter: `int` (optional) if retries was necessary because of connection fails Following rfc3261 specification, an exponential backoff factor is used. More specifically: backoff = 0.5 T1 = 500ms max_retries_call = 7 time_to_live_call = 64 seconds max_retries_sms = 11 time_to_live_sms ~ 2 hours """ retries = self.retries_params[self.media_type] - counter backoff_factor = self.retries_params['bo_factor'] * 0.5 * 2 ** counter status_forcelist = tuple([status for status in range(500, 600)]) session = None session = session or requests.Session() retry = Retry( total=retries, read=retries, connect=retries, backoff_factor=backoff_factor, status_forcelist=status_forcelist) adapter = HTTPAdapter(max_retries=retry) session.mount('http://', adapter) session.mount('https://', adapter) return session def send_notification(self, got401=False) -> dict: """ Send a Firebase push notification """ if self.error: self.log_error() return {'code': 500, 'body': {}, 'reason': 'Internal server error'} n_retries, backoff_factor = self.retries_params(self.wp_request.media_type) counter = 0 error = False code = 500 reason = "" body = None response = None while counter <= n_retries: self.log_request(path=self.pns.url_push) try: response = requests.post(self.pns.url_push, self.payload, headers=self.headers) break except requests.exceptions.RequestException as e: error = True reason = f'connection failed: {e}' counter += 1 timer = backoff_factor * (2 ** (counter - 1)) time.sleep(timer) if counter == n_retries: reason = "maximum retries reached" elif error: try: response = self.requests_retry_session(counter). \ post(self.pns.url_push, self.payload, headers=self.headers) except Exception as x: level = 'error' msg = f"outgoing {self.platform.title()} response for " \ f"{self.request_id}, push failed: " \ f"an error occurred in {x.__class__.__name__}" log_event(loggers=self.loggers, msg=msg, level=level) try: body = response.__dict__ except (TypeError, ValueError): code = 500 reason = 'cannot parse response body' body = {} else: reason = body.get('reason') code = response.status_code for k in ('raw', 'request', 'connection', 'cookies', 'elapsed'): try: del body[k] except KeyError: pass except TypeError: break body = json.dumps(fix_non_serializable_types(body)) if isinstance(body, str): body = json.loads(body) if code == 200: try: failure = body['_content']['failure'] except KeyError: pass else: if failure == 1: reason = body['_content']['results'][0]['error'] code = 410 elif code == 404: try: payload_code = body['_content']['error']['code'] except KeyError: pass else: if payload_code == 404: code = 410 keys = list(body.keys()) for key in keys: if not body[key]: del body[key] results = {'body': body, 'code': code, 'reason': reason, 'url': self.pns.url_push, 'call_id': self.wp_request.call_id, 'token': self.token } self.results = results self.log_results() # Request is missing required authentication credential. # Expected OAuth 2 access token, login cookie or other valid authentication # credential. UNAUTHENTICATED code = results.get('code') reason = results.get('reason') if not got401 and code == 401 and reason == 'Unauthorized': delta = datetime.now() - self.pns.last_update_token level = 'warn' msg = f"outgoing {self.platform.title()} response for request " \ f"{self.request_id} need a new access token - " \ f"server will refresh it and try again" log_event(loggers=self.loggers, msg=msg, level=level, to_file=True) # retry with a new Fireplace access token self.pns.access_token = self.pns.set_access_token() level = 'warn' msg = f"outgoing {self.platform.title()} response for request " \ f"{self.request_id} a new access token {self.pns.access_token} was generated - " \ f"trying again" log_event(loggers=self.loggers, msg=msg, level=level, to_file=True) self.results = self.send_notification(got401=True) return results