Initial commit
This commit is contained in:
+14
@@ -0,0 +1,14 @@
|
||||
**sublime**
|
||||
*.pyc
|
||||
dist
|
||||
*.egg-info
|
||||
dist
|
||||
MANIFEST
|
||||
build
|
||||
.eggs
|
||||
.coverage
|
||||
.vscode/
|
||||
env
|
||||
flake8.out
|
||||
pylint.out
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
Copyright (c) 2020 PostHog (part of Hiberly Inc)
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
include README.md
|
||||
@@ -0,0 +1,13 @@
|
||||
test:
|
||||
pylint --rcfile=.pylintrc --reports=y --exit-zero analytics | tee pylint.out
|
||||
flake8 --max-complexity=10 --statistics analytics > flake8.out || true
|
||||
coverage run --branch --include=analytics/\* --omit=*/test* setup.py test
|
||||
|
||||
release:
|
||||
python setup.py sdist bdist_wheel
|
||||
twine upload dist/*
|
||||
|
||||
e2e_test:
|
||||
.buildscripts/e2e.sh
|
||||
|
||||
.PHONY: test release e2e_test
|
||||
@@ -0,0 +1,84 @@
|
||||
# PostHog Python
|
||||
|
||||
Official PostHog Python library to capture and send events to any PostHhog instance (including PostHog.com).
|
||||
|
||||
This library uses an internal queue to make calls non-blocking and fast. It also batches requests and flushes asynchronously, making it perfect to use in any part of your web app or other server side application that needs performance.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install posthog
|
||||
```
|
||||
|
||||
In your app, import the posthog library and set your api key **before** making any calls.
|
||||
|
||||
```python
|
||||
import posthog
|
||||
|
||||
posthog.write_key = 'YOUR API KEY'
|
||||
```
|
||||
|
||||
You can find your key in the /setup page in PostHog.
|
||||
|
||||
To debug, you can set debug mode.
|
||||
```python
|
||||
posthog.debug = True
|
||||
```
|
||||
|
||||
## Making calls
|
||||
|
||||
### Capture
|
||||
|
||||
Capture allows you to capture anything a user does within your system, which you can later use in PostHog to find patterns in usage, work out which features to improve or where people are giving up.
|
||||
|
||||
A `capture` call requires
|
||||
- `distinct id` which uniquely identifies your user
|
||||
- `event name` to make sure
|
||||
- We recommend using [verb] [noun], like `movie played` or `movie updated` to easily identify what your events mean later on.
|
||||
|
||||
Optionally you can submit
|
||||
- `properties`, which can be a dict with any information you'd like to add
|
||||
|
||||
For example:
|
||||
```python
|
||||
posthog.capture('distinct id', 'movie played', {'movie_id': '123', 'category': 'romcom'})
|
||||
```
|
||||
|
||||
### Identify
|
||||
Identify lets you add metadata on your users so you can more easily identify who they are in PostHog, and even do things like segment users by these properties.
|
||||
|
||||
An `identify` call requires
|
||||
- `distinct id` which uniquely identifies your user
|
||||
- `properties` with a dict with any key: value pairs
|
||||
|
||||
For example:
|
||||
```python
|
||||
posthog.capture('distinct id', {
|
||||
'email': 'dwayne@gmail.com',
|
||||
'name': 'Dwayne Johnson'
|
||||
})
|
||||
```
|
||||
|
||||
The most obvious place to make this call is whenever a user signs up, or when they update their information.
|
||||
|
||||
### Alias
|
||||
|
||||
To marry up whatever a user does before they sign up or log in with what they do after you need to make an alias call. This will allow you to answer questions like "Which marketing channels leads to users churning after a month?" or "What do users do on our website before signing up?"
|
||||
|
||||
In a purely back-end implementation, this means whenever an anonymous user does something, you'll want to send a session ID ([Django](https://stackoverflow.com/questions/526179/in-django-how-can-i-find-out-the-request-session-sessionid-and-use-it-as-a-vari), [Flask](https://stackoverflow.com/questions/15156132/flask-login-how-to-get-session-id)) with the capture call. Then, when that users signs up, you want to do an alias call with the session ID and the newly created user ID.
|
||||
|
||||
The same concept applies for when a user logs in.
|
||||
|
||||
An `alias` call requires
|
||||
- `previous distinct id` the unique ID of the user before
|
||||
- `distinct id` the current unique id
|
||||
|
||||
For example:
|
||||
```python
|
||||
posthog.alias('anonymous session id', 'distinct id')
|
||||
```
|
||||
|
||||
|
||||
## Thank you
|
||||
|
||||
This library is largely based on the `analytics-python` package.
|
||||
@@ -0,0 +1,8 @@
|
||||
Releasing
|
||||
=========
|
||||
|
||||
1. Update `VERSION` in `posthog/version.py` to the new version.
|
||||
2. Update the `HISTORY.md` for the impending release.
|
||||
3. `git commit -am "Release X.Y.Z."` (where X.Y.Z is the new version)
|
||||
4. `git tag -a X.Y.Z -m "Version X.Y.Z"` (where X.Y.Z is the new version).
|
||||
5. `make release`.
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/bin/sh
|
||||
|
||||
set -e
|
||||
|
||||
python ./simulator.py "$@"
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
# PostHog Python library example
|
||||
|
||||
# Import the library
|
||||
import posthog
|
||||
|
||||
# You can find this key on the /setup page in PostHog
|
||||
posthog.write_key = '<your key>'
|
||||
|
||||
# Where you host PostHog, with no trailing /.
|
||||
# You can remove this line if you're using posthog.com
|
||||
posthog.host = 'http://127.0.0.1:8000'
|
||||
|
||||
# Capture an event
|
||||
posthog.capture('distinct_id', 'event', {'property1': 'value', 'property2': 'value'})
|
||||
|
||||
# Alias a previous distinct id with a new one
|
||||
posthog.alias('distinct_id', 'new_distinct_id')
|
||||
|
||||
# Add properties to the person
|
||||
posthog.identify('distinct_id', {'email': 'something@something.com'})
|
||||
@@ -0,0 +1,72 @@
|
||||
|
||||
from posthog.version import VERSION
|
||||
from posthog.client import Client
|
||||
|
||||
__version__ = VERSION
|
||||
|
||||
"""Settings."""
|
||||
write_key = None
|
||||
host = None
|
||||
on_error = None
|
||||
debug = False
|
||||
send = True
|
||||
sync_mode = False
|
||||
|
||||
default_client = None
|
||||
|
||||
|
||||
def capture(*args, **kwargs):
|
||||
"""Send a capture call."""
|
||||
_proxy('capture', *args, **kwargs)
|
||||
|
||||
def identify(*args, **kwargs):
|
||||
"""Send a identify call."""
|
||||
_proxy('identify', *args, **kwargs)
|
||||
|
||||
|
||||
def group(*args, **kwargs):
|
||||
"""Send a group call."""
|
||||
_proxy('group', *args, **kwargs)
|
||||
|
||||
|
||||
def alias(*args, **kwargs):
|
||||
"""Send a alias call."""
|
||||
_proxy('alias', *args, **kwargs)
|
||||
|
||||
|
||||
def page(*args, **kwargs):
|
||||
"""Send a page call."""
|
||||
_proxy('page', *args, **kwargs)
|
||||
|
||||
|
||||
def screen(*args, **kwargs):
|
||||
"""Send a screen call."""
|
||||
_proxy('screen', *args, **kwargs)
|
||||
|
||||
|
||||
def flush():
|
||||
"""Tell the client to flush."""
|
||||
_proxy('flush')
|
||||
|
||||
|
||||
def join():
|
||||
"""Block program until the client clears the queue"""
|
||||
_proxy('join')
|
||||
|
||||
|
||||
def shutdown():
|
||||
"""Flush all messages and cleanly shutdown the client"""
|
||||
_proxy('flush')
|
||||
_proxy('join')
|
||||
|
||||
|
||||
def _proxy(method, *args, **kwargs):
|
||||
"""Create an analytics client if one doesn't exist and send to it."""
|
||||
global default_client
|
||||
if not default_client:
|
||||
default_client = Client(write_key, host=host, debug=debug,
|
||||
on_error=on_error, send=send,
|
||||
sync_mode=sync_mode)
|
||||
|
||||
fn = getattr(default_client, method)
|
||||
fn(*args, **kwargs)
|
||||
@@ -0,0 +1,287 @@
|
||||
from datetime import datetime
|
||||
from uuid import uuid4
|
||||
import logging
|
||||
import numbers
|
||||
import atexit
|
||||
|
||||
from dateutil.tz import tzutc
|
||||
from six import string_types
|
||||
|
||||
from posthog.utils import guess_timezone, clean
|
||||
from posthog.consumer import Consumer
|
||||
from posthog.request import post
|
||||
from posthog.version import VERSION
|
||||
|
||||
try:
|
||||
import queue
|
||||
except ImportError:
|
||||
import Queue as queue
|
||||
|
||||
|
||||
ID_TYPES = (numbers.Number, string_types)
|
||||
|
||||
|
||||
class Client(object):
|
||||
"""Create a new PostHog client."""
|
||||
log = logging.getLogger('posthog')
|
||||
|
||||
def __init__(self, write_key=None, host=None, debug=False,
|
||||
max_queue_size=10000, send=True, on_error=None, flush_at=100,
|
||||
flush_interval=0.5, gzip=False, max_retries=3,
|
||||
sync_mode=False, timeout=15, thread=1):
|
||||
require('write_key', write_key, string_types)
|
||||
|
||||
self.queue = queue.Queue(max_queue_size)
|
||||
self.write_key = write_key
|
||||
self.on_error = on_error
|
||||
self.debug = debug
|
||||
self.send = send
|
||||
self.sync_mode = sync_mode
|
||||
self.host = host
|
||||
self.gzip = gzip
|
||||
self.timeout = timeout
|
||||
|
||||
if debug:
|
||||
self.log.setLevel(logging.DEBUG)
|
||||
|
||||
if sync_mode:
|
||||
self.consumers = None
|
||||
else:
|
||||
# On program exit, allow the consumer thread to exit cleanly.
|
||||
# This prevents exceptions and a messy shutdown when the
|
||||
# interpreter is destroyed before the daemon thread finishes
|
||||
# execution. However, it is *not* the same as flushing the queue!
|
||||
# To guarantee all messages have been delivered, you'll still need
|
||||
# to call flush().
|
||||
if send:
|
||||
atexit.register(self.join)
|
||||
for n in range(thread):
|
||||
self.consumers = []
|
||||
consumer = Consumer(
|
||||
self.queue, write_key, host=host, on_error=on_error,
|
||||
flush_at=flush_at, flush_interval=flush_interval,
|
||||
gzip=gzip, retries=max_retries, timeout=timeout,
|
||||
)
|
||||
self.consumers.append(consumer)
|
||||
|
||||
# if we've disabled sending, just don't start the consumer
|
||||
if send:
|
||||
consumer.start()
|
||||
|
||||
def identify(self, distinct_id=None, traits=None, context=None, timestamp=None,
|
||||
message_id=None):
|
||||
traits = traits or {}
|
||||
context = context or {}
|
||||
require('distinct_id', distinct_id, ID_TYPES)
|
||||
require('traits', traits, dict)
|
||||
|
||||
msg = {
|
||||
'timestamp': timestamp,
|
||||
'context': context,
|
||||
'type': 'identify',
|
||||
'distinct_id': distinct_id,
|
||||
'$set': traits,
|
||||
'event': '$identify',
|
||||
'messageId': message_id,
|
||||
}
|
||||
|
||||
return self._enqueue(msg)
|
||||
|
||||
def capture(self, distinct_id=None, event=None, properties=None, context=None,
|
||||
timestamp=None, message_id=None):
|
||||
properties = properties or {}
|
||||
context = context or {}
|
||||
require('distinct_id', distinct_id, ID_TYPES)
|
||||
require('properties', properties, dict)
|
||||
require('event', event, string_types)
|
||||
|
||||
msg = {
|
||||
'properties': properties,
|
||||
'timestamp': timestamp,
|
||||
'context': context,
|
||||
'distinct_id': distinct_id,
|
||||
'type': 'capture',
|
||||
'event': event,
|
||||
'messageId': message_id,
|
||||
}
|
||||
|
||||
return self._enqueue(msg)
|
||||
|
||||
def alias(self, previous_id=None, distinct_id=None, context=None,
|
||||
timestamp=None, message_id=None):
|
||||
context = context or {}
|
||||
|
||||
require('previous_id', previous_id, ID_TYPES)
|
||||
require('distinct_id', distinct_id, ID_TYPES)
|
||||
|
||||
msg = {
|
||||
'properties': {
|
||||
'distinct_id': previous_id,
|
||||
'alias': distinct_id,
|
||||
},
|
||||
'timestamp': timestamp,
|
||||
'context': context,
|
||||
'type': 'alias',
|
||||
'event': '$create_alias'
|
||||
}
|
||||
|
||||
return self._enqueue(msg)
|
||||
|
||||
def group(self, distinct_id=None, group_id=None, traits=None, context=None,
|
||||
timestamp=None, message_id=None):
|
||||
traits = traits or {}
|
||||
context = context or {}
|
||||
|
||||
require('distinct_id', distinct_id, ID_TYPES)
|
||||
require('group_id', group_id, ID_TYPES)
|
||||
require('traits', traits, dict)
|
||||
|
||||
msg = {
|
||||
'timestamp': timestamp,
|
||||
'groupId': group_id,
|
||||
'context': context,
|
||||
'distinct_id': distinct_id,
|
||||
'traits': traits,
|
||||
'type': 'group',
|
||||
'messageId': message_id,
|
||||
}
|
||||
|
||||
return self._enqueue(msg)
|
||||
|
||||
def page(self, distinct_id=None, category=None, name=None, properties=None,
|
||||
context=None, timestamp=None, message_id=None):
|
||||
properties = properties or {}
|
||||
context = context or {}
|
||||
|
||||
require('distinct_id', distinct_id, ID_TYPES)
|
||||
require('properties', properties, dict)
|
||||
|
||||
if name:
|
||||
require('name', name, string_types)
|
||||
if category:
|
||||
require('category', category, string_types)
|
||||
|
||||
msg = {
|
||||
'properties': properties,
|
||||
'timestamp': timestamp,
|
||||
'category': category,
|
||||
'context': context,
|
||||
'distinct_id': distinct_id,
|
||||
'type': 'page',
|
||||
'name': name,
|
||||
'messageId': message_id,
|
||||
}
|
||||
|
||||
return self._enqueue(msg)
|
||||
|
||||
def screen(self, distinct_id=None, category=None, name=None, properties=None,
|
||||
context=None, timestamp=None, message_id=None):
|
||||
properties = properties or {}
|
||||
context = context or {}
|
||||
|
||||
require('distinct_id', distinct_id, ID_TYPES)
|
||||
require('properties', properties, dict)
|
||||
|
||||
if name:
|
||||
require('name', name, string_types)
|
||||
if category:
|
||||
require('category', category, string_types)
|
||||
|
||||
msg = {
|
||||
|
||||
'properties': properties,
|
||||
'timestamp': timestamp,
|
||||
'category': category,
|
||||
'context': context,
|
||||
'distinct_id': distinct_id,
|
||||
'type': 'screen',
|
||||
'name': name,
|
||||
'messageId': message_id,
|
||||
}
|
||||
|
||||
return self._enqueue(msg)
|
||||
|
||||
def _enqueue(self, msg):
|
||||
"""Push a new `msg` onto the queue, return `(success, msg)`"""
|
||||
timestamp = msg['timestamp']
|
||||
if timestamp is None:
|
||||
timestamp = datetime.utcnow().replace(tzinfo=tzutc())
|
||||
message_id = msg.get('messageId')
|
||||
if message_id is None:
|
||||
message_id = uuid4()
|
||||
|
||||
require('type', msg['type'], string_types)
|
||||
require('timestamp', timestamp, datetime)
|
||||
require('context', msg['context'], dict)
|
||||
|
||||
# add common
|
||||
timestamp = guess_timezone(timestamp)
|
||||
msg['timestamp'] = timestamp.isoformat()
|
||||
msg['messageId'] = stringify_id(message_id)
|
||||
msg['library'] = 'posthog-python'
|
||||
msg['library_version'] = VERSION
|
||||
msg['api_key'] = self.write_key
|
||||
|
||||
msg['distinct_id'] = stringify_id(msg.get('distinct_id', None))
|
||||
|
||||
msg = clean(msg)
|
||||
self.log.debug('queueing: %s', msg)
|
||||
|
||||
# if send is False, return msg as if it was successfully queued
|
||||
if not self.send:
|
||||
return True, msg
|
||||
|
||||
if self.sync_mode:
|
||||
self.log.debug('enqueued with blocking %s.', msg['type'])
|
||||
post(self.host, gzip=self.gzip,
|
||||
timeout=self.timeout, batch=[msg])
|
||||
|
||||
return True, msg
|
||||
|
||||
try:
|
||||
self.queue.put(msg, block=False)
|
||||
self.log.debug('enqueued %s.', msg['type'])
|
||||
return True, msg
|
||||
except queue.Full:
|
||||
self.log.warning('analytics-python queue is full')
|
||||
return False, msg
|
||||
|
||||
def flush(self):
|
||||
"""Forces a flush from the internal queue to the server"""
|
||||
queue = self.queue
|
||||
size = queue.qsize()
|
||||
queue.join()
|
||||
# Note that this message may not be precise, because of threading.
|
||||
self.log.debug('successfully flushed about %s items.', size)
|
||||
|
||||
def join(self):
|
||||
"""Ends the consumer thread once the queue is empty.
|
||||
Blocks execution until finished
|
||||
"""
|
||||
for consumer in self.consumers:
|
||||
consumer.pause()
|
||||
try:
|
||||
consumer.join()
|
||||
except RuntimeError:
|
||||
# consumer thread has not started
|
||||
pass
|
||||
|
||||
def shutdown(self):
|
||||
"""Flush all messages and cleanly shutdown the client"""
|
||||
self.flush()
|
||||
self.join()
|
||||
|
||||
|
||||
def require(name, field, data_type):
|
||||
"""Require that the named `field` has the right `data_type`"""
|
||||
if not isinstance(field, data_type):
|
||||
msg = '{0} must have {1}, got: {2}'.format(name, data_type, field)
|
||||
raise AssertionError(msg)
|
||||
|
||||
|
||||
def stringify_id(val):
|
||||
if val is None:
|
||||
return None
|
||||
if isinstance(val, string_types):
|
||||
return val
|
||||
return str(val)
|
||||
@@ -0,0 +1,134 @@
|
||||
import logging
|
||||
from threading import Thread
|
||||
import monotonic
|
||||
import backoff
|
||||
import json
|
||||
|
||||
from posthog.request import post, APIError, DatetimeSerializer
|
||||
|
||||
try:
|
||||
from queue import Empty
|
||||
except ImportError:
|
||||
from Queue import Empty
|
||||
|
||||
MAX_MSG_SIZE = 32 << 10
|
||||
|
||||
# Our servers only accept batches less than 500KB. Here limit is set slightly
|
||||
# lower to leave space for extra data that will be added later, eg. "sentAt".
|
||||
BATCH_SIZE_LIMIT = 475000
|
||||
|
||||
|
||||
class Consumer(Thread):
|
||||
"""Consumes the messages from the client's queue."""
|
||||
log = logging.getLogger('posthog')
|
||||
|
||||
def __init__(self, queue, write_key, flush_at=100, host=None,
|
||||
on_error=None, flush_interval=0.5, gzip=False, retries=10,
|
||||
timeout=15):
|
||||
"""Create a consumer thread."""
|
||||
Thread.__init__(self)
|
||||
# Make consumer a daemon thread so that it doesn't block program exit
|
||||
self.daemon = True
|
||||
self.flush_at = flush_at
|
||||
self.flush_interval = flush_interval
|
||||
self.write_key = write_key
|
||||
self.host = host
|
||||
self.on_error = on_error
|
||||
self.queue = queue
|
||||
self.gzip = gzip
|
||||
# It's important to set running in the constructor: if we are asked to
|
||||
# pause immediately after construction, we might set running to True in
|
||||
# run() *after* we set it to False in pause... and keep running
|
||||
# forever.
|
||||
self.running = True
|
||||
self.retries = retries
|
||||
self.timeout = timeout
|
||||
|
||||
def run(self):
|
||||
"""Runs the consumer."""
|
||||
self.log.debug('consumer is running...')
|
||||
while self.running:
|
||||
self.upload()
|
||||
|
||||
self.log.debug('consumer exited.')
|
||||
|
||||
def pause(self):
|
||||
"""Pause the consumer."""
|
||||
self.running = False
|
||||
|
||||
def upload(self):
|
||||
"""Upload the next batch of items, return whether successful."""
|
||||
success = False
|
||||
batch = self.next()
|
||||
if len(batch) == 0:
|
||||
return False
|
||||
|
||||
try:
|
||||
self.request(batch)
|
||||
success = True
|
||||
except Exception as e:
|
||||
self.log.error('error uploading: %s', e)
|
||||
success = False
|
||||
if self.on_error:
|
||||
self.on_error(e, batch)
|
||||
finally:
|
||||
# mark items as acknowledged from queue
|
||||
for item in batch:
|
||||
self.queue.task_done()
|
||||
return success
|
||||
|
||||
def next(self):
|
||||
"""Return the next batch of items to upload."""
|
||||
queue = self.queue
|
||||
items = []
|
||||
|
||||
start_time = monotonic.monotonic()
|
||||
total_size = 0
|
||||
|
||||
while len(items) < self.flush_at:
|
||||
elapsed = monotonic.monotonic() - start_time
|
||||
if elapsed >= self.flush_interval:
|
||||
break
|
||||
try:
|
||||
item = queue.get(
|
||||
block=True, timeout=self.flush_interval - elapsed)
|
||||
item_size = len(json.dumps(
|
||||
item, cls=DatetimeSerializer).encode())
|
||||
if item_size > MAX_MSG_SIZE:
|
||||
self.log.error(
|
||||
'Item exceeds 32kb limit, dropping. (%s)', str(item))
|
||||
continue
|
||||
items.append(item)
|
||||
total_size += item_size
|
||||
if total_size >= BATCH_SIZE_LIMIT:
|
||||
self.log.debug(
|
||||
'hit batch size limit (size: %d)', total_size)
|
||||
break
|
||||
except Empty:
|
||||
break
|
||||
|
||||
return items
|
||||
|
||||
def request(self, batch):
|
||||
"""Attempt to upload the batch and retry before raising an error """
|
||||
|
||||
def fatal_exception(exc):
|
||||
if isinstance(exc, APIError):
|
||||
# retry on server errors and client errors
|
||||
# with 429 status code (rate limited),
|
||||
# don't retry on other client errors
|
||||
return (400 <= exc.status < 500) and exc.status != 429
|
||||
else:
|
||||
# retry on all other errors (eg. network)
|
||||
return False
|
||||
|
||||
@backoff.on_exception(
|
||||
backoff.expo,
|
||||
Exception,
|
||||
max_tries=self.retries + 1,
|
||||
giveup=fatal_exception)
|
||||
def send_request():
|
||||
post(self.host, gzip=self.gzip,
|
||||
timeout=self.timeout, batch=batch)
|
||||
|
||||
send_request()
|
||||
@@ -0,0 +1,69 @@
|
||||
from datetime import date, datetime
|
||||
from dateutil.tz import tzutc
|
||||
import logging
|
||||
import json
|
||||
from gzip import GzipFile
|
||||
from requests.auth import HTTPBasicAuth
|
||||
from requests import sessions
|
||||
from io import BytesIO
|
||||
|
||||
from posthog.version import VERSION
|
||||
from posthog.utils import remove_trailing_slash
|
||||
|
||||
_session = sessions.Session()
|
||||
|
||||
|
||||
def post(host=None, gzip=False, timeout=15, **kwargs):
|
||||
"""Post the `kwargs` to the API"""
|
||||
log = logging.getLogger('posthog')
|
||||
body = kwargs
|
||||
body["sentAt"] = datetime.utcnow().replace(tzinfo=tzutc()).isoformat()
|
||||
url = remove_trailing_slash(host or 'https://t.posthog.com') + '/batch/'
|
||||
data = json.dumps(body, cls=DatetimeSerializer)
|
||||
log.debug('making request: %s', data)
|
||||
headers = {
|
||||
'Content-Type': 'application/json',
|
||||
'User-Agent': 'analytics-python/' + VERSION
|
||||
}
|
||||
if gzip:
|
||||
headers['Content-Encoding'] = 'gzip'
|
||||
buf = BytesIO()
|
||||
with GzipFile(fileobj=buf, mode='w') as gz:
|
||||
# 'data' was produced by json.dumps(),
|
||||
# whose default encoding is utf-8.
|
||||
gz.write(data.encode('utf-8'))
|
||||
data = buf.getvalue()
|
||||
|
||||
res = _session.post(url, data=data,
|
||||
headers=headers, timeout=timeout)
|
||||
|
||||
if res.status_code == 200:
|
||||
log.debug('data uploaded successfully')
|
||||
return res
|
||||
|
||||
try:
|
||||
payload = res.json()
|
||||
log.debug('received response: %s', payload)
|
||||
raise APIError(res.status_code, payload['code'], payload['message'])
|
||||
except ValueError:
|
||||
raise APIError(res.status_code, 'unknown', res.text)
|
||||
|
||||
|
||||
class APIError(Exception):
|
||||
|
||||
def __init__(self, status, code, message):
|
||||
self.message = message
|
||||
self.status = status
|
||||
self.code = code
|
||||
|
||||
def __str__(self):
|
||||
msg = "[PostHog] {0}: {1} ({2})"
|
||||
return msg.format(self.code, self.message, self.status)
|
||||
|
||||
|
||||
class DatetimeSerializer(json.JSONEncoder):
|
||||
def default(self, obj):
|
||||
if isinstance(obj, (date, datetime)):
|
||||
return obj.isoformat()
|
||||
|
||||
return json.JSONEncoder.default(self, obj)
|
||||
@@ -0,0 +1,14 @@
|
||||
import unittest
|
||||
import pkgutil
|
||||
import logging
|
||||
import sys
|
||||
|
||||
|
||||
def all_names():
|
||||
for _, modname, _ in pkgutil.iter_modules(__path__):
|
||||
yield 'analytics.test.' + modname
|
||||
|
||||
|
||||
def all():
|
||||
logging.basicConfig(stream=sys.stderr)
|
||||
return unittest.defaultTestLoader.loadTestsFromNames(all_names())
|
||||
@@ -0,0 +1,316 @@
|
||||
from datetime import date, datetime
|
||||
import unittest
|
||||
import six
|
||||
import mock
|
||||
import time
|
||||
|
||||
from posthog.version import VERSION
|
||||
from posthoglient import Client
|
||||
|
||||
|
||||
class TestClient(unittest.TestCase):
|
||||
|
||||
def fail(self, e, batch):
|
||||
"""Mark the failure handler"""
|
||||
self.failed = True
|
||||
|
||||
def setUp(self):
|
||||
self.failed = False
|
||||
self.client = Client('testsecret', on_error=self.fail)
|
||||
|
||||
def test_requires_write_key(self):
|
||||
self.assertRaises(AssertionError, Client)
|
||||
|
||||
def test_empty_flush(self):
|
||||
self.client.flush()
|
||||
|
||||
def test_basic_track(self):
|
||||
client = self.client
|
||||
success, msg = client.track('distinct_id', 'python test event')
|
||||
client.flush()
|
||||
self.assertTrue(success)
|
||||
self.assertFalse(self.failed)
|
||||
|
||||
self.assertEqual(msg['event'], 'python test event')
|
||||
self.assertTrue(isinstance(msg['timestamp'], str))
|
||||
self.assertTrue(isinstance(msg['messageId'], str))
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['properties'], {})
|
||||
self.assertEqual(msg['type'], 'track')
|
||||
|
||||
def test_stringifies_distinct_id(self):
|
||||
# A large number that loses precision in node:
|
||||
# node -e "console.log(157963456373623802 + 1)" > 157963456373623800
|
||||
client = self.client
|
||||
success, msg = client.track(
|
||||
distinct_id=157963456373623802, event='python test event')
|
||||
client.flush()
|
||||
self.assertTrue(success)
|
||||
self.assertFalse(self.failed)
|
||||
|
||||
self.assertEqual(msg['distinct_id'], '157963456373623802')
|
||||
|
||||
def test_advanced_track(self):
|
||||
client = self.client
|
||||
success, msg = client.track(
|
||||
'distinct_id', 'python test event', {'property': 'value'},
|
||||
{'ip': '192.168.0.1'}, datetime(2014, 9, 3),
|
||||
'messageId')
|
||||
|
||||
self.assertTrue(success)
|
||||
|
||||
self.assertEqual(msg['timestamp'], '2014-09-03T00:00:00+00:00')
|
||||
self.assertEqual(msg['properties'], {'property': 'value'})
|
||||
self.assertEqual(msg['context']['ip'], '192.168.0.1')
|
||||
self.assertEqual(msg['event'], 'python test event')
|
||||
self.assertEqual(msg['context']['library'], {
|
||||
'name': 'analytics-python',
|
||||
'version': VERSION
|
||||
})
|
||||
self.assertEqual(msg['messageId'], 'messageId')
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['type'], 'track')
|
||||
|
||||
def test_basic_identify(self):
|
||||
client = self.client
|
||||
success, msg = client.identify('distinct_id', {'trait': 'value'})
|
||||
client.flush()
|
||||
self.assertTrue(success)
|
||||
self.assertFalse(self.failed)
|
||||
|
||||
self.assertEqual(msg['traits'], {'trait': 'value'})
|
||||
self.assertTrue(isinstance(msg['timestamp'], str))
|
||||
self.assertTrue(isinstance(msg['messageId'], str))
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['type'], 'identify')
|
||||
|
||||
def test_advanced_identify(self):
|
||||
client = self.client
|
||||
success, msg = client.identify(
|
||||
'distinct_id', {'trait': 'value'}, {'ip': '192.168.0.1'},
|
||||
datetime(2014, 9, 3), 'messageId')
|
||||
|
||||
self.assertTrue(success)
|
||||
|
||||
self.assertEqual(msg['timestamp'], '2014-09-03T00:00:00+00:00')
|
||||
self.assertEqual(msg['context']['ip'], '192.168.0.1')
|
||||
self.assertEqual(msg['traits'], {'trait': 'value'})
|
||||
self.assertEqual(msg['context']['library'], {
|
||||
'name': 'analytics-python',
|
||||
'version': VERSION
|
||||
})
|
||||
self.assertTrue(isinstance(msg['timestamp'], str))
|
||||
self.assertEqual(msg['messageId'], 'messageId')
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['type'], 'identify')
|
||||
|
||||
def test_basic_group(self):
|
||||
client = self.client
|
||||
success, msg = client.group('distinct_id', 'groupId')
|
||||
client.flush()
|
||||
self.assertTrue(success)
|
||||
self.assertFalse(self.failed)
|
||||
|
||||
self.assertEqual(msg['groupId'], 'groupId')
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['type'], 'group')
|
||||
|
||||
def test_advanced_group(self):
|
||||
client = self.client
|
||||
success, msg = client.group(
|
||||
'distinct_id', 'groupId', {'trait': 'value'}, {'ip': '192.168.0.1'},
|
||||
datetime(2014, 9, 3), 'messageId')
|
||||
|
||||
self.assertTrue(success)
|
||||
|
||||
self.assertEqual(msg['timestamp'], '2014-09-03T00:00:00+00:00')
|
||||
self.assertEqual(msg['context']['ip'], '192.168.0.1')
|
||||
self.assertEqual(msg['traits'], {'trait': 'value'})
|
||||
self.assertEqual(msg['context']['library'], {
|
||||
'name': 'analytics-python',
|
||||
'version': VERSION
|
||||
})
|
||||
self.assertTrue(isinstance(msg['timestamp'], str))
|
||||
self.assertEqual(msg['messageId'], 'messageId')
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['type'], 'group')
|
||||
|
||||
def test_basic_alias(self):
|
||||
client = self.client
|
||||
success, msg = client.alias('previousId', 'distinct_id')
|
||||
client.flush()
|
||||
self.assertTrue(success)
|
||||
self.assertFalse(self.failed)
|
||||
self.assertEqual(msg['previousId'], 'previousId')
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
|
||||
def test_basic_page(self):
|
||||
client = self.client
|
||||
success, msg = client.page('distinct_id', name='name')
|
||||
self.assertFalse(self.failed)
|
||||
client.flush()
|
||||
self.assertTrue(success)
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['type'], 'page')
|
||||
self.assertEqual(msg['name'], 'name')
|
||||
|
||||
def test_advanced_page(self):
|
||||
client = self.client
|
||||
success, msg = client.page(
|
||||
'distinct_id', 'category', 'name', {'property': 'value'},
|
||||
{'ip': '192.168.0.1'}, datetime(2014, 9, 3), 'messageId')
|
||||
|
||||
self.assertTrue(success)
|
||||
|
||||
self.assertEqual(msg['timestamp'], '2014-09-03T00:00:00+00:00')
|
||||
self.assertEqual(msg['context']['ip'], '192.168.0.1')
|
||||
self.assertEqual(msg['properties'], {'property': 'value'})
|
||||
self.assertEqual(msg['context']['library'], {
|
||||
'name': 'analytics-python',
|
||||
'version': VERSION
|
||||
})
|
||||
self.assertEqual(msg['category'], 'category')
|
||||
self.assertTrue(isinstance(msg['timestamp'], str))
|
||||
self.assertEqual(msg['messageId'], 'messageId')
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['type'], 'page')
|
||||
self.assertEqual(msg['name'], 'name')
|
||||
|
||||
def test_basic_screen(self):
|
||||
client = self.client
|
||||
success, msg = client.screen('distinct_id', name='name')
|
||||
client.flush()
|
||||
self.assertTrue(success)
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['type'], 'screen')
|
||||
self.assertEqual(msg['name'], 'name')
|
||||
|
||||
def test_advanced_screen(self):
|
||||
client = self.client
|
||||
success, msg = client.screen(
|
||||
'distinct_id', 'category', 'name', {'property': 'value'},
|
||||
{'ip': '192.168.0.1'}, datetime(2014, 9, 3), 'messageId')
|
||||
|
||||
self.assertTrue(success)
|
||||
|
||||
self.assertEqual(msg['timestamp'], '2014-09-03T00:00:00+00:00')
|
||||
self.assertEqual(msg['context']['ip'], '192.168.0.1')
|
||||
self.assertEqual(msg['properties'], {'property': 'value'})
|
||||
self.assertEqual(msg['context']['library'], {
|
||||
'name': 'analytics-python',
|
||||
'version': VERSION
|
||||
})
|
||||
self.assertTrue(isinstance(msg['timestamp'], str))
|
||||
self.assertEqual(msg['messageId'], 'messageId')
|
||||
self.assertEqual(msg['category'], 'category')
|
||||
self.assertEqual(msg['distinct_id'], 'distinct_id')
|
||||
self.assertEqual(msg['type'], 'screen')
|
||||
self.assertEqual(msg['name'], 'name')
|
||||
|
||||
def test_flush(self):
|
||||
client = self.client
|
||||
# set up the consumer with more requests than a single batch will allow
|
||||
for i in range(1000):
|
||||
success, msg = client.identify('distinct_id', {'trait': 'value'})
|
||||
# We can't reliably assert that the queue is non-empty here; that's
|
||||
# a race condition. We do our best to load it up though.
|
||||
client.flush()
|
||||
# Make sure that the client queue is empty after flushing
|
||||
self.assertTrue(client.queue.empty())
|
||||
|
||||
def test_shutdown(self):
|
||||
client = self.client
|
||||
# set up the consumer with more requests than a single batch will allow
|
||||
for i in range(1000):
|
||||
success, msg = client.identify('distinct_id', {'trait': 'value'})
|
||||
client.shutdown()
|
||||
# we expect two things after shutdown:
|
||||
# 1. client queue is empty
|
||||
# 2. consumer thread has stopped
|
||||
self.assertTrue(client.queue.empty())
|
||||
for consumer in client.consumers:
|
||||
self.assertFalse(consumer.is_alive())
|
||||
|
||||
def test_synchronous(self):
|
||||
client = Client('testsecret', sync_mode=True)
|
||||
|
||||
success, message = client.identify('distinct_id')
|
||||
self.assertFalse(client.consumers)
|
||||
self.assertTrue(client.queue.empty())
|
||||
self.assertTrue(success)
|
||||
|
||||
def test_overflow(self):
|
||||
client = Client('testsecret', max_queue_size=1)
|
||||
# Ensure consumer thread is no longer uploading
|
||||
client.join()
|
||||
|
||||
for i in range(10):
|
||||
client.identify('distinct_id')
|
||||
|
||||
success, msg = client.identify('distinct_id')
|
||||
# Make sure we are informed that the queue is at capacity
|
||||
self.assertFalse(success)
|
||||
|
||||
def test_success_on_invalid_write_key(self):
|
||||
client = Client('bad_key', on_error=self.fail)
|
||||
client.track('distinct_id', 'event')
|
||||
client.flush()
|
||||
self.assertFalse(self.failed)
|
||||
|
||||
def test_unicode(self):
|
||||
Client(six.u('unicode_key'))
|
||||
|
||||
def test_numeric_distinct_id(self):
|
||||
self.client.track(1234, 'python event')
|
||||
self.client.flush()
|
||||
self.assertFalse(self.failed)
|
||||
|
||||
def test_debug(self):
|
||||
Client('bad_key', debug=True)
|
||||
|
||||
def test_identify_with_date_object(self):
|
||||
client = self.client
|
||||
success, msg = client.identify(
|
||||
'distinct_id',
|
||||
{
|
||||
'birthdate': date(1981, 2, 2),
|
||||
},
|
||||
)
|
||||
client.flush()
|
||||
self.assertTrue(success)
|
||||
self.assertFalse(self.failed)
|
||||
|
||||
self.assertEqual(msg['traits'], {'birthdate': date(1981, 2, 2)})
|
||||
|
||||
def test_gzip(self):
|
||||
client = Client('testsecret', on_error=self.fail, gzip=True)
|
||||
for _ in range(10):
|
||||
client.identify('distinct_id', {'trait': 'value'})
|
||||
client.flush()
|
||||
self.assertFalse(self.failed)
|
||||
|
||||
def test_user_defined_flush_at(self):
|
||||
client = Client('testsecret', on_error=self.fail,
|
||||
flush_at=10, flush_interval=3)
|
||||
|
||||
def mock_post_fn(*args, **kwargs):
|
||||
self.assertEquals(len(kwargs['batch']), 10)
|
||||
|
||||
# the post function should be called 2 times, with a batch size of 10
|
||||
# each time.
|
||||
with mock.patch('analytics.consumer.post', side_effect=mock_post_fn) \
|
||||
as mock_post:
|
||||
for _ in range(20):
|
||||
client.identify('distinct_id', {'trait': 'value'})
|
||||
time.sleep(1)
|
||||
self.assertEquals(mock_post.call_count, 2)
|
||||
|
||||
def test_user_defined_timeout(self):
|
||||
client = Client('testsecret', timeout=10)
|
||||
for consumer in client.consumers:
|
||||
self.assertEquals(consumer.timeout, 10)
|
||||
|
||||
def test_default_timeout_15(self):
|
||||
client = Client('testsecret')
|
||||
for consumer in client.consumers:
|
||||
self.assertEquals(consumer.timeout, 15)
|
||||
@@ -0,0 +1,198 @@
|
||||
import unittest
|
||||
import mock
|
||||
import time
|
||||
import json
|
||||
|
||||
try:
|
||||
from queue import Queue
|
||||
except ImportError:
|
||||
from Queue import Queue
|
||||
|
||||
from posthog.consumer import Consumer, MAX_MSG_SIZE
|
||||
from posthogequest import APIError
|
||||
|
||||
|
||||
class TestConsumer(unittest.TestCase):
|
||||
|
||||
def test_next(self):
|
||||
q = Queue()
|
||||
consumer = Consumer(q, '')
|
||||
q.put(1)
|
||||
next = consumer.next()
|
||||
self.assertEqual(next, [1])
|
||||
|
||||
def test_next_limit(self):
|
||||
q = Queue()
|
||||
flush_at = 50
|
||||
consumer = Consumer(q, '', flush_at)
|
||||
for i in range(10000):
|
||||
q.put(i)
|
||||
next = consumer.next()
|
||||
self.assertEqual(next, list(range(flush_at)))
|
||||
|
||||
def test_dropping_oversize_msg(self):
|
||||
q = Queue()
|
||||
consumer = Consumer(q, '')
|
||||
oversize_msg = {'m': 'x' * MAX_MSG_SIZE}
|
||||
q.put(oversize_msg)
|
||||
next = consumer.next()
|
||||
self.assertEqual(next, [])
|
||||
self.assertTrue(q.empty())
|
||||
|
||||
def test_upload(self):
|
||||
q = Queue()
|
||||
consumer = Consumer(q, 'testsecret')
|
||||
track = {
|
||||
'type': 'track',
|
||||
'event': 'python event',
|
||||
'distinct_id': 'distinct_id'
|
||||
}
|
||||
q.put(track)
|
||||
success = consumer.upload()
|
||||
self.assertTrue(success)
|
||||
|
||||
def test_flush_interval(self):
|
||||
# Put _n_ items in the queue, pausing a little bit more than
|
||||
# _flush_interval_ after each one.
|
||||
# The consumer should upload _n_ times.
|
||||
q = Queue()
|
||||
flush_interval = 0.3
|
||||
consumer = Consumer(q, 'testsecret', flush_at=10,
|
||||
flush_interval=flush_interval)
|
||||
with mock.patch('analytics.consumer.post') as mock_post:
|
||||
consumer.start()
|
||||
for i in range(0, 3):
|
||||
track = {
|
||||
'type': 'track',
|
||||
'event': 'python event %d' % i,
|
||||
'distinct_id': 'distinct_id'
|
||||
}
|
||||
q.put(track)
|
||||
time.sleep(flush_interval * 1.1)
|
||||
self.assertEqual(mock_post.call_count, 3)
|
||||
|
||||
def test_multiple_uploads_per_interval(self):
|
||||
# Put _flush_at*2_ items in the queue at once, then pause for
|
||||
# _flush_interval_. The consumer should upload 2 times.
|
||||
q = Queue()
|
||||
flush_interval = 0.5
|
||||
flush_at = 10
|
||||
consumer = Consumer(q, 'testsecret', flush_at=flush_at,
|
||||
flush_interval=flush_interval)
|
||||
with mock.patch('analytics.consumer.post') as mock_post:
|
||||
consumer.start()
|
||||
for i in range(0, flush_at * 2):
|
||||
track = {
|
||||
'type': 'track',
|
||||
'event': 'python event %d' % i,
|
||||
'distinct_id': 'distinct_id'
|
||||
}
|
||||
q.put(track)
|
||||
time.sleep(flush_interval * 1.1)
|
||||
self.assertEqual(mock_post.call_count, 2)
|
||||
|
||||
def test_request(self):
|
||||
consumer = Consumer(None, 'testsecret')
|
||||
track = {
|
||||
'type': 'track',
|
||||
'event': 'python event',
|
||||
'distinct_id': 'distinct_id'
|
||||
}
|
||||
consumer.request([track])
|
||||
|
||||
def _test_request_retry(self, consumer,
|
||||
expected_exception, exception_count):
|
||||
|
||||
def mock_post(*args, **kwargs):
|
||||
mock_post.call_count += 1
|
||||
if mock_post.call_count <= exception_count:
|
||||
raise expected_exception
|
||||
mock_post.call_count = 0
|
||||
|
||||
with mock.patch('analytics.consumer.post',
|
||||
mock.Mock(side_effect=mock_post)):
|
||||
track = {
|
||||
'type': 'track',
|
||||
'event': 'python event',
|
||||
'distinct_id': 'distinct_id'
|
||||
}
|
||||
# request() should succeed if the number of exceptions raised is
|
||||
# less than the retries paramater.
|
||||
if exception_count <= consumer.retries:
|
||||
consumer.request([track])
|
||||
else:
|
||||
# if exceptions are raised more times than the retries
|
||||
# parameter, we expect the exception to be returned to
|
||||
# the caller.
|
||||
try:
|
||||
consumer.request([track])
|
||||
except type(expected_exception) as exc:
|
||||
self.assertEqual(exc, expected_exception)
|
||||
else:
|
||||
self.fail(
|
||||
"request() should raise an exception if still failing "
|
||||
"after %d retries" % consumer.retries)
|
||||
|
||||
def test_request_retry(self):
|
||||
# we should retry on general errors
|
||||
consumer = Consumer(None, 'testsecret')
|
||||
self._test_request_retry(consumer, Exception('generic exception'), 2)
|
||||
|
||||
# we should retry on server errors
|
||||
consumer = Consumer(None, 'testsecret')
|
||||
self._test_request_retry(consumer, APIError(
|
||||
500, 'code', 'Internal Server Error'), 2)
|
||||
|
||||
# we should retry on HTTP 429 errors
|
||||
consumer = Consumer(None, 'testsecret')
|
||||
self._test_request_retry(consumer, APIError(
|
||||
429, 'code', 'Too Many Requests'), 2)
|
||||
|
||||
# we should NOT retry on other client errors
|
||||
consumer = Consumer(None, 'testsecret')
|
||||
api_error = APIError(400, 'code', 'Client Errors')
|
||||
try:
|
||||
self._test_request_retry(consumer, api_error, 1)
|
||||
except APIError:
|
||||
pass
|
||||
else:
|
||||
self.fail('request() should not retry on client errors')
|
||||
|
||||
# test for number of exceptions raise > retries value
|
||||
consumer = Consumer(None, 'testsecret', retries=3)
|
||||
self._test_request_retry(consumer, APIError(
|
||||
500, 'code', 'Internal Server Error'), 3)
|
||||
|
||||
def test_pause(self):
|
||||
consumer = Consumer(None, 'testsecret')
|
||||
consumer.pause()
|
||||
self.assertFalse(consumer.running)
|
||||
|
||||
def test_max_batch_size(self):
|
||||
q = Queue()
|
||||
consumer = Consumer(
|
||||
q, 'testsecret', flush_at=100000, flush_interval=3)
|
||||
track = {
|
||||
'type': 'track',
|
||||
'event': 'python event',
|
||||
'distinct_id': 'distinct_id'
|
||||
}
|
||||
msg_size = len(json.dumps(track).encode())
|
||||
# number of messages in a maximum-size batch
|
||||
n_msgs = int(475000 / msg_size)
|
||||
|
||||
def mock_post_fn(_, data, **kwargs):
|
||||
res = mock.Mock()
|
||||
res.status_code = 200
|
||||
self.assertTrue(len(data.encode()) < 500000,
|
||||
'batch size (%d) exceeds 500KB limit'
|
||||
% len(data.encode()))
|
||||
return res
|
||||
|
||||
with mock.patch('analytics.request._session.post',
|
||||
side_effect=mock_post_fn) as mock_post:
|
||||
consumer.start()
|
||||
for _ in range(0, n_msgs + 2):
|
||||
q.put(track)
|
||||
q.join()
|
||||
self.assertEquals(mock_post.call_count, 2)
|
||||
@@ -0,0 +1,49 @@
|
||||
import unittest
|
||||
|
||||
import analytics
|
||||
|
||||
|
||||
class TestModule(unittest.TestCase):
|
||||
|
||||
def failed(self):
|
||||
self.failed = True
|
||||
|
||||
def setUp(self):
|
||||
self.failed = False
|
||||
analytics.write_key = 'testsecret'
|
||||
analytics.on_error = self.failed
|
||||
|
||||
def test_no_write_key(self):
|
||||
analytics.write_key = None
|
||||
self.assertRaises(Exception, analytics.track)
|
||||
|
||||
def test_no_host(self):
|
||||
analytics.host = None
|
||||
self.assertRaises(Exception, analytics.track)
|
||||
|
||||
def test_track(self):
|
||||
analytics.track('distinct_id', 'python module event')
|
||||
analytics.flush()
|
||||
|
||||
def test_identify(self):
|
||||
analytics.identify('distinct_id', {'email': 'user@email.com'})
|
||||
analytics.flush()
|
||||
|
||||
def test_group(self):
|
||||
analytics.group('distinct_id', 'groupId')
|
||||
analytics.flush()
|
||||
|
||||
def test_alias(self):
|
||||
analytics.alias('previousId', 'distinct_id')
|
||||
analytics.flush()
|
||||
|
||||
def test_page(self):
|
||||
analytics.page('distinct_id')
|
||||
analytics.flush()
|
||||
|
||||
def test_screen(self):
|
||||
analytics.screen('distinct_id')
|
||||
analytics.flush()
|
||||
|
||||
def test_flush(self):
|
||||
analytics.flush()
|
||||
@@ -0,0 +1,53 @@
|
||||
from datetime import datetime, date
|
||||
import unittest
|
||||
import json
|
||||
import requests
|
||||
|
||||
from posthogequest import post, DatetimeSerializer
|
||||
|
||||
|
||||
class TestRequests(unittest.TestCase):
|
||||
|
||||
def test_valid_request(self):
|
||||
res = post(batch=[{
|
||||
'distinct_id': 'distinct_id',
|
||||
'event': 'python event',
|
||||
'type': 'track'
|
||||
}])
|
||||
self.assertEqual(res.status_code, 200)
|
||||
|
||||
def test_invalid_request_error(self):
|
||||
self.assertRaises(Exception, post, 'testsecret',
|
||||
'https://t.posthog.com', False, '[{]')
|
||||
|
||||
def test_invalid_host(self):
|
||||
self.assertRaises(Exception, post, 'testsecret',
|
||||
't.posthog.com/', batch=[])
|
||||
|
||||
def test_datetime_serialization(self):
|
||||
data = {'created': datetime(2012, 3, 4, 5, 6, 7, 891011)}
|
||||
result = json.dumps(data, cls=DatetimeSerializer)
|
||||
self.assertEqual(result, '{"created": "2012-03-04T05:06:07.891011"}')
|
||||
|
||||
def test_date_serialization(self):
|
||||
today = date.today()
|
||||
data = {'created': today}
|
||||
result = json.dumps(data, cls=DatetimeSerializer)
|
||||
expected = '{"created": "%s"}' % today.isoformat()
|
||||
self.assertEqual(result, expected)
|
||||
|
||||
def test_should_not_timeout(self):
|
||||
res = post(batch=[{
|
||||
'distinct_id': 'distinct_id',
|
||||
'event': 'python event',
|
||||
'type': 'track'
|
||||
}], timeout=15)
|
||||
self.assertEqual(res.status_code, 200)
|
||||
|
||||
def test_should_timeout(self):
|
||||
with self.assertRaises(requests.ReadTimeout):
|
||||
post(batch=[{
|
||||
'distinct_id': 'distinct_id',
|
||||
'event': 'python event',
|
||||
'type': 'track'
|
||||
}], timeout=0.0001)
|
||||
@@ -0,0 +1,78 @@
|
||||
from datetime import date, datetime, timedelta
|
||||
from decimal import Decimal
|
||||
import unittest
|
||||
|
||||
from dateutil.tz import tzutc
|
||||
import six
|
||||
|
||||
from posthog import utils
|
||||
|
||||
|
||||
class TestUtils(unittest.TestCase):
|
||||
|
||||
def test_timezone_utils(self):
|
||||
now = datetime.now()
|
||||
utcnow = datetime.now(tz=tzutc())
|
||||
self.assertTrue(utils.is_naive(now))
|
||||
self.assertFalse(utils.is_naive(utcnow))
|
||||
|
||||
fixed = utils.guess_timezone(now)
|
||||
self.assertFalse(utils.is_naive(fixed))
|
||||
|
||||
shouldnt_be_edited = utils.guess_timezone(utcnow)
|
||||
self.assertEqual(utcnow, shouldnt_be_edited)
|
||||
|
||||
def test_clean(self):
|
||||
simple = {
|
||||
'decimal': Decimal('0.142857'),
|
||||
'unicode': six.u('woo'),
|
||||
'date': datetime.now(),
|
||||
'long': 200000000,
|
||||
'integer': 1,
|
||||
'float': 2.0,
|
||||
'bool': True,
|
||||
'str': 'woo',
|
||||
'none': None
|
||||
}
|
||||
|
||||
complicated = {
|
||||
'exception': Exception('This should show up'),
|
||||
'timedelta': timedelta(microseconds=20),
|
||||
'list': [1, 2, 3]
|
||||
}
|
||||
|
||||
combined = dict(simple.items())
|
||||
combined.update(complicated.items())
|
||||
|
||||
pre_clean_keys = combined.keys()
|
||||
|
||||
utils.clean(combined)
|
||||
self.assertEqual(combined.keys(), pre_clean_keys)
|
||||
|
||||
def test_clean_with_dates(self):
|
||||
dict_with_dates = {
|
||||
'birthdate': date(1980, 1, 1),
|
||||
'registration': datetime.utcnow(),
|
||||
}
|
||||
self.assertEqual(dict_with_dates, utils.clean(dict_with_dates))
|
||||
|
||||
def test_bytes(self):
|
||||
if six.PY3:
|
||||
item = bytes(10)
|
||||
else:
|
||||
item = bytearray(10)
|
||||
|
||||
utils.clean(item)
|
||||
|
||||
def test_clean_fn(self):
|
||||
cleaned = utils.clean({'fn': lambda x: x, 'number': 4})
|
||||
self.assertEqual(cleaned['number'], 4)
|
||||
# TODO: fixme, different behavior on python 2 and 3
|
||||
if 'fn' in cleaned:
|
||||
self.assertEqual(cleaned['fn'], None)
|
||||
|
||||
def test_remove_slash(self):
|
||||
self.assertEqual('http://posthog.io',
|
||||
utils.remove_trailing_slash('http://posthog.io/'))
|
||||
self.assertEqual('http://posthog.io',
|
||||
utils.remove_trailing_slash('http://posthog.io'))
|
||||
@@ -0,0 +1,87 @@
|
||||
from dateutil.tz import tzlocal, tzutc
|
||||
from datetime import date, datetime
|
||||
from decimal import Decimal
|
||||
import logging
|
||||
import numbers
|
||||
|
||||
import six
|
||||
|
||||
log = logging.getLogger('posthog')
|
||||
|
||||
|
||||
def is_naive(dt):
|
||||
"""Determines if a given datetime.datetime is naive."""
|
||||
return dt.tzinfo is None or dt.tzinfo.utcoffset(dt) is None
|
||||
|
||||
|
||||
def total_seconds(delta):
|
||||
"""Determines total seconds with python < 2.7 compat."""
|
||||
# http://stackoverflow.com/questions/3694835/python-2-6-5-divide-timedelta-with-timedelta
|
||||
return (delta.microseconds
|
||||
+ (delta.seconds + delta.days * 24 * 3600) * 1e6) / 1e6
|
||||
|
||||
|
||||
def guess_timezone(dt):
|
||||
"""Attempts to convert a naive datetime to an aware datetime."""
|
||||
if is_naive(dt):
|
||||
# attempts to guess the datetime.datetime.now() local timezone
|
||||
# case, and then defaults to utc
|
||||
delta = datetime.now() - dt
|
||||
if total_seconds(delta) < 5:
|
||||
# this was created using datetime.datetime.now()
|
||||
# so we are in the local timezone
|
||||
return dt.replace(tzinfo=tzlocal())
|
||||
else:
|
||||
# at this point, the best we can do is guess UTC
|
||||
return dt.replace(tzinfo=tzutc())
|
||||
|
||||
return dt
|
||||
|
||||
|
||||
def remove_trailing_slash(host):
|
||||
if host.endswith('/'):
|
||||
return host[:-1]
|
||||
return host
|
||||
|
||||
|
||||
def clean(item):
|
||||
if isinstance(item, Decimal):
|
||||
return float(item)
|
||||
elif isinstance(item, (six.string_types, bool, numbers.Number, datetime,
|
||||
date, type(None))):
|
||||
return item
|
||||
elif isinstance(item, (set, list, tuple)):
|
||||
return _clean_list(item)
|
||||
elif isinstance(item, dict):
|
||||
return _clean_dict(item)
|
||||
else:
|
||||
return _coerce_unicode(item)
|
||||
|
||||
|
||||
def _clean_list(list_):
|
||||
return [clean(item) for item in list_]
|
||||
|
||||
|
||||
def _clean_dict(dict_):
|
||||
data = {}
|
||||
for k, v in six.iteritems(dict_):
|
||||
try:
|
||||
data[k] = clean(v)
|
||||
except TypeError:
|
||||
log.warning(
|
||||
'Dictionary values must be serializeable to '
|
||||
'JSON "%s" value %s of type %s is unsupported.',
|
||||
k, v, type(v),
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
def _coerce_unicode(cmplx):
|
||||
try:
|
||||
item = cmplx.decode("utf-8", "strict")
|
||||
except AttributeError as exception:
|
||||
item = ":".join(exception)
|
||||
item.decode("utf-8", "strict")
|
||||
log.warning('Error decoding: %s', item)
|
||||
return None
|
||||
return item
|
||||
@@ -0,0 +1 @@
|
||||
VERSION = '1.0.0'
|
||||
@@ -0,0 +1,63 @@
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
try:
|
||||
from setuptools import setup
|
||||
except ImportError:
|
||||
from distutils.core import setup
|
||||
|
||||
# Don't import analytics-python module here, since deps may not be installed
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'posthog'))
|
||||
from version import VERSION
|
||||
|
||||
long_description = '''
|
||||
PostHog is developer-friendly, self-hosted product analytics. posthog-python is the python package.
|
||||
'''
|
||||
|
||||
install_requires = [
|
||||
"requests>=2.7,<3.0",
|
||||
"six>=1.5",
|
||||
"monotonic>=1.5",
|
||||
"backoff==1.6.0",
|
||||
"python-dateutil>2.1"
|
||||
]
|
||||
|
||||
tests_require = [
|
||||
"mock>=2.0.0"
|
||||
]
|
||||
|
||||
setup(
|
||||
name='posthog',
|
||||
version=VERSION,
|
||||
url='https://github.com/posthog/posthog-python',
|
||||
author='Posthog',
|
||||
author_email='hey@posthog.com',
|
||||
maintainer='PostHog',
|
||||
maintainer_email='hey@posthog.com',
|
||||
test_suite='posthog.test.all',
|
||||
packages=['posthog', 'posthog.test'],
|
||||
license='MIT License',
|
||||
install_requires=install_requires,
|
||||
tests_require=tests_require,
|
||||
description='Integrate PostHog into any python application.',
|
||||
long_description=long_description,
|
||||
classifiers=[
|
||||
"Development Status :: 5 - Production/Stable",
|
||||
"Intended Audience :: Developers",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Operating System :: OS Independent",
|
||||
"Programming Language :: Python",
|
||||
"Programming Language :: Python :: 2",
|
||||
"Programming Language :: Python :: 2.6",
|
||||
"Programming Language :: Python :: 2.7",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.2",
|
||||
"Programming Language :: Python :: 3.3",
|
||||
"Programming Language :: Python :: 3.4",
|
||||
"Programming Language :: Python :: 3.5",
|
||||
"Programming Language :: Python :: 3.6",
|
||||
"Programming Language :: Python :: 3.7",
|
||||
"Programming Language :: Python :: 3.8",
|
||||
],
|
||||
)
|
||||
@@ -0,0 +1,99 @@
|
||||
import analytics
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
|
||||
__name__ = 'simulator.py'
|
||||
__version__ = '0.0.1'
|
||||
__description__ = 'scripting simulator'
|
||||
|
||||
|
||||
def json_hash(str):
|
||||
if str:
|
||||
return json.loads(str)
|
||||
|
||||
# analytics -method=<method> -posthog-write-key=<posthogWriteKey> [options]
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(description='send a posthog message')
|
||||
|
||||
parser.add_argument('--writeKey', help='the posthog writeKey')
|
||||
parser.add_argument('--type', help='The posthog message type')
|
||||
|
||||
parser.add_argument('--distinct_id', help='the user id to send the event as')
|
||||
parser.add_argument(
|
||||
'--anonymousId', help='the anonymous user id to send the event as')
|
||||
parser.add_argument(
|
||||
'--context', help='additional context for the event (JSON-encoded)')
|
||||
|
||||
parser.add_argument('--event', help='the event name to send with the event')
|
||||
parser.add_argument(
|
||||
'--properties', help='the event properties to send (JSON-encoded)')
|
||||
|
||||
parser.add_argument(
|
||||
'--name', help='name of the screen or page to send with the message')
|
||||
|
||||
parser.add_argument(
|
||||
'--traits', help='the identify/group traits to send (JSON-encoded)')
|
||||
|
||||
parser.add_argument('--groupId', help='the group id')
|
||||
|
||||
options = parser.parse_args()
|
||||
|
||||
|
||||
def failed(status, msg):
|
||||
raise Exception(msg)
|
||||
|
||||
|
||||
def track():
|
||||
analytics.track(options.distinct_id, options.event, anonymous_id=options.anonymousId,
|
||||
properties=json_hash(options.properties), context=json_hash(options.context))
|
||||
|
||||
|
||||
def page():
|
||||
analytics.page(options.distinct_id, name=options.name, anonymous_id=options.anonymousId,
|
||||
properties=json_hash(options.properties), context=json_hash(options.context))
|
||||
|
||||
|
||||
def screen():
|
||||
analytics.screen(options.distinct_id, name=options.name, anonymous_id=options.anonymousId,
|
||||
properties=json_hash(options.properties), context=json_hash(options.context))
|
||||
|
||||
|
||||
def identify():
|
||||
analytics.identify(options.distinct_id, anonymous_id=options.anonymousId,
|
||||
traits=json_hash(options.traits), context=json_hash(options.context))
|
||||
|
||||
|
||||
def group():
|
||||
analytics.group(options.distinct_id, options.groupId, json_hash(options.traits),
|
||||
json_hash(options.context), anonymous_id=options.anonymousId)
|
||||
|
||||
|
||||
def unknown():
|
||||
print()
|
||||
|
||||
|
||||
analytics.write_key = options.writeKey
|
||||
analytics.on_error = failed
|
||||
analytics.debug = True
|
||||
|
||||
log = logging.getLogger('posthog')
|
||||
ch = logging.StreamHandler()
|
||||
ch.setLevel(logging.DEBUG)
|
||||
log.addHandler(ch)
|
||||
|
||||
switcher = {
|
||||
"track": track,
|
||||
"page": page,
|
||||
"screen": screen,
|
||||
"identify": identify,
|
||||
"group": group
|
||||
}
|
||||
|
||||
func = switcher.get(options.type)
|
||||
if func:
|
||||
func()
|
||||
analytics.shutdown()
|
||||
else:
|
||||
print("Invalid Message Type " + options.type)
|
||||
Reference in New Issue
Block a user