Use flags_by key for faster access

* chore(flags): Update type hints for newer mypy versions

* black

.github/workflows/ci.yml

@@ -33,6 +33,10 @@ jobs:
             - name: Check formatting with black
               run: |
                   black --check .
+            
+            - name: Lint with flake8
+              run: |
+                  flake8 posthog --ignore E501
 
             - name: Check import order with isort
               run: |
@@ -58,4 +62,4 @@ jobs:
 
             - name: Run posthog tests
               run: |
-                  python setup.py test
+                  pytest --verbose --timeout=30

CHANGELOG.md

@@ -1,3 +1,147 @@
+## 3.5.0 - 2024-02-29
+
+1. - Adds a new `feature_flags_request_timeout_seconds` timeout parameter for feature flags which defaults to 3 seconds, updated from the default 10s for all other API calls.
+
+## 3.4.2 - 2024-02-20
+
+1. Add `historical_migration` option for bulk migration to PostHog Cloud.
+
+## 3.4.1 - 2024-02-09
+
+1. Use new hosts for event capture as well
+
+## 3.4.0 - 2024-02-05
+
+1. Point given hosts to new ingestion hosts
+
+## 3.3.4 - 2024-01-30
+
+1. Update type hints for module variables to work with newer versions of mypy
+
+## 3.3.3 - 2024-01-26
+
+1. Remove new relative date operators, combine into regular date operators
+
+## 3.3.2 - 2024-01-19
+
+1. Return success/failure with all capture calls from module functions
+
+
+## 3.3.1 - 2024-01-10
+
+1. Make sure we don't override any existing feature flag properties when adding locally evaluated feature flag properties.
+
+## 3.3.0 - 2024-01-09
+
+1. When local evaluation is enabled, we automatically add flag information to all events sent to PostHog, whenever possible. This makes it easier to use these events in experiments.
+
+## 3.2.0 - 2024-01-09
+
+1. Numeric property handling for feature flags now does the expected: When passed in a number, we do a numeric comparison. When passed in a string, we do a string comparison. Previously, we always did a string comparison.
+2. Add support for relative date operators for local evaluation.
+
+## 3.1.0 - 2023-12-04
+
+1. Increase maximum event size and batch size
+
+## 3.0.2 - 2023-08-17
+
+1. Returns the current flag property with $feature_flag_called events, to make it easier to use in experiments
+
+## 3.0.1 - 2023-04-21
+
+1. Restore how feature flags work when the client library is disabled: All requests return `None` and no events are sent when the client is disabled.
+2. Add a `feature_flag_definitions()` debug option, which returns currently loaded feature flag definitions. You can use this to more cleverly decide when to request local evaluation of feature flags.
+## 3.0.0 - 2023-04-14
+
+Breaking change:
+
+All events by default now send the `$geoip_disable` property to disable geoip lookup in app. This is because usually we don't
+want to update person properties to take the server's location.
+
+The same now happens for feature flag requests, where we discard the IP address of the server for matching on geoip properties like city, country, continent.
+
+To restore previous behaviour, you can set the default to False like so:
+
+```python
+posthog.disable_geoip = False
+
+# // and if using client instantiation:
+posthog = Posthog('api_key', disable_geoip=False)
+
+```
+
+## 2.5.0 - 2023-04-10
+
+1. Add option for instantiating separate client object
+
+## 2.4.2 - 2023-03-30
+
+1. Update backoff dependency for posthoganalytics package to be the same as posthog package
+
+## 2.4.1 - 2023-03-17
+
+1. Removes accidental print call left in for decide response
+
+## 2.4.0 - 2023-03-14
+
+1. Support evaluating all cohorts in feature flags for local evaluation
+
+## 2.3.1 - 2023-02-07
+
+1. Log instead of raise error on posthog personal api key errors
+2. Remove upper bound on backoff dependency
+
+
+## 2.3.0 - 2023-01-31
+
+1. Add support for returning payloads of matched feature flags
+
+## 2.2.0 - 2022-11-14
+
+Changes:
+
+1. Add support for feature flag variant overrides with local evaluation
+
+## 2.1.2 - 2022-09-15
+
+Changes:
+
+1. Fixes issues with date comparison.
+## 2.1.1 - 2022-09-14
+
+Changes:
+
+1. Feature flags local evaluation now supports date property filters as well. Accepts both strings and datetime objects.
+
+## 2.1.0 - 2022-08-11
+
+Changes:
+
+1. Feature flag defaults have been removed
+2. Setup logging only when debug mode is enabled.
+## 2.0.1 - 2022-08-04
+
+- Make poll_interval configurable
+- Add `send_feature_flag_events` parameter to feature flag calls, which determine whether the `$feature_flag_called` event should be sent or not.
+- Add `only_evaluate_locally` parameter to feature flag calls, which determines whether the feature flag should only be evaluated locally or not.
+
+## 2.0.0 - 2022-08-02
+
+Breaking changes:
+
+1. The minimum version requirement for PostHog servers is now 1.38. If you're using PostHog Cloud, you satisfy this requirement automatically.
+2. Feature flag defaults apply only when there's an error fetching feature flag results. Earlier, if the default was set to `True`, even if a flag resolved to `False`, the default would override this.
+**Note: These are removed in 2.0.2**
+3. Feature flag remote evaluation doesn't require a personal API key.
+
+New Changes:
+
+1. You can now evaluate feature flags locally (i.e. without sending a request to your PostHog servers) by setting a personal API key, and passing in groups and person properties to `is_feature_enabled` and `get_feature_flag` calls.
+2. Introduces a `get_all_flags` method that returns all feature flags. This is useful for when you want to seed your frontend with some initial flags, given a user ID.
+
+
+
 ## 1.4.9 - 2022-06-13
 - Support for sending feature flags with capture calls
 

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2020 PostHog (part of Hiberly Inc)
+Copyright (c) 2023 PostHog (part of Hiberly Inc)
 
 Copyright (c) 2013 Segment Inc. friends@segment.com
 

README.md

@@ -39,7 +39,7 @@ Then navigate to `http://127.0.0.1:8080/sentry-debug/` and you should get an eve
 
 ### Releasing Versions
 
-Updated are released using GitHub Actions: after bumping `version.py` in `master`, go to [our release workflow's page](https://github.com/PostHog/posthog-python/actions/workflows/release.yaml) and dispatch it manually, using workflow from `master`.
+Updated are released using GitHub Actions: after bumping `version.py` in `master` and adding to `CHANGELOG.md`, go to [our release workflow's page](https://github.com/PostHog/posthog-python/actions/workflows/release.yaml) and dispatch it manually, using workflow from `master`.
 
 ## Questions?
 

example.py

@@ -5,25 +5,45 @@ import time
 
 import posthog
 
+posthog.debug = True
+
 # You can find this key on the /setup page in PostHog
-posthog.project_api_key = ""
-posthog.personal_api_key = ""
+posthog.project_api_key = "phc_gtWmTq3Pgl06u4sZY3TRcoQfp42yfuXHKoe8ZVSR6Kh"
+posthog.personal_api_key = "phx_fiRCOQkTA3o2ePSdLrFDAILLHjMu2Mv52vUi8MNruIm"
 
 # Where you host PostHog, with no trailing /.
 # You can remove this line if you're using posthog.com
 posthog.host = "http://localhost:8000"
+posthog.poll_interval = 10
+
+print(
+    posthog.feature_enabled(
+        "person-on-events-enabled",
+        "12345",
+        groups={"organization": str("0182ee91-8ef7-0000-4cb9-fedc5f00926a")},
+        group_properties={
+            "organization": {
+                "id": "0182ee91-8ef7-0000-4cb9-fedc5f00926a",
+                "created_at": "2022-06-30 11:44:52.984121+00:00",
+            }
+        },
+        only_evaluate_locally=True,
+    )
+)
+
 
 # Capture an event
 posthog.capture("distinct_id", "event", {"property1": "value", "property2": "value"}, send_feature_flags=True)
 
 print(posthog.feature_enabled("beta-feature", "distinct_id"))
-print(posthog.feature_enabled("beta-feature", "distinct_id", groups={"company": "id:5"}))
-
-print("sleeping")
-time.sleep(5)
+print(posthog.feature_enabled("beta-feature-groups", "distinct_id", groups={"company": "id:5"}))
 
 print(posthog.feature_enabled("beta-feature", "distinct_id"))
 
+# get payload
+print(posthog.get_feature_flag_payload("beta-feature", "distinct_id"))
+print(posthog.get_all_flags_and_payloads("distinct_id"))
+exit()
 # # Alias a previous distinct id with a new one
 
 posthog.alias("distinct_id", "new_distinct_id")
@@ -42,7 +62,6 @@ posthog.group_identify("company", "id:5", {"employees": 11})
 # properties set only once to the person
 posthog.set_once("new_distinct_id", {"self_serve_signup": True})
 
-time.sleep(3)
 
 posthog.set_once(
     "new_distinct_id", {"self_serve_signup": False}
@@ -51,4 +70,32 @@ posthog.set_once(
 posthog.set("new_distinct_id", {"current_browser": "Chrome"})
 posthog.set("new_distinct_id", {"current_browser": "Firefox"})
 
-# posthog.shutdown()
+
+# #############################################################################
+# Make sure you have a personal API key for the examples below
+
+# Local Evaluation
+
+# If flag has City=Sydney, this call doesn't go to `/decide`
+print(posthog.feature_enabled("test-flag", "distinct_id_random_22", person_properties={"$geoip_city_name": "Sydney"}))
+
+print(
+    posthog.feature_enabled(
+        "test-flag",
+        "distinct_id_random_22",
+        person_properties={"$geoip_city_name": "Sydney"},
+        only_evaluate_locally=True,
+    )
+)
+
+
+print(posthog.get_all_flags("distinct_id_random_22"))
+print(posthog.get_all_flags("distinct_id_random_22", only_evaluate_locally=True))
+print(
+    posthog.get_all_flags(
+        "distinct_id_random_22", person_properties={"$geoip_city_name": "Sydney"}, only_evaluate_locally=True
+    )
+)
+
+
+posthog.shutdown()

posthog/__init__.py

@@ -1,4 +1,5 @@
-from typing import Callable, Dict, Optional
+import datetime  # noqa: F401
+from typing import Callable, Dict, Optional, Tuple  # noqa: F401
 
 from posthog.client import Client
 from posthog.version import VERSION
@@ -6,17 +7,20 @@ from posthog.version import VERSION
 __version__ = VERSION
 
 """Settings."""
-api_key = None  # type: str
-host = None  # type: str
-on_error = None  # type: Callable
+api_key = None  # type: Optional[str]
+host = None  # type: Optional[str]
+on_error = None  # type: Optional[Callable]
 debug = False  # type: bool
 send = True  # type: bool
 sync_mode = False  # type: bool
 disabled = False  # type: bool
-personal_api_key = None  # type: str
-project_api_key = None  # type: str
+personal_api_key = None  # type: Optional[str]
+project_api_key = None  # type: Optional[str]
+poll_interval = 30  # type: int
+disable_geoip = True  # type: bool
+feature_flags_request_timeout_seconds = 3  # type: int
 
-default_client = None
+default_client = None  # type: Optional[Client]
 
 
 def capture(
@@ -28,8 +32,9 @@ def capture(
     uuid=None,  # type: Optional[str]
     groups=None,  # type: Optional[Dict]
     send_feature_flags=False,
+    disable_geoip=None,  # type: Optional[bool]
 ):
-    # type: (...) -> None
+    # type: (...) -> Tuple[bool, dict]
     """
     Capture allows you to capture anything a user does within your system, which you can later use in PostHog to find patterns in usage, work out which features to improve or where people are giving up.
 
@@ -50,7 +55,7 @@ def capture(
     posthog.capture('distinct id', 'purchase', groups={'company': 'id:5'})
     ```
     """
-    _proxy(
+    return _proxy(
         "capture",
         distinct_id=distinct_id,
         event=event,
@@ -60,6 +65,7 @@ def capture(
         uuid=uuid,
         groups=groups,
         send_feature_flags=send_feature_flags,
+        disable_geoip=disable_geoip,
     )
 
 
@@ -69,8 +75,9 @@ def identify(
     context=None,  # type: Optional[Dict]
     timestamp=None,  # type: Optional[datetime.datetime]
     uuid=None,  # type: Optional[str]
+    disable_geoip=None,  # type: Optional[bool]
 ):
-    # type: (...) -> None
+    # type: (...) -> Tuple[bool, dict]
     """
     Identify lets you add metadata on your users so you can more easily identify who they are in PostHog, and even do things like segment users by these properties.
 
@@ -86,24 +93,26 @@ def identify(
     })
     ```
     """
-    _proxy(
+    return _proxy(
         "identify",
         distinct_id=distinct_id,
         properties=properties,
         context=context,
         timestamp=timestamp,
         uuid=uuid,
+        disable_geoip=disable_geoip,
     )
 
 
 def set(
-    distinct_id,  # type: str,
+    distinct_id,  # type: str
     properties=None,  # type: Optional[Dict]
     context=None,  # type: Optional[Dict]
     timestamp=None,  # type: Optional[datetime.datetime]
     uuid=None,  # type: Optional[str]
+    disable_geoip=None,  # type: Optional[bool]
 ):
-    # type: (...) -> None
+    # type: (...) -> Tuple[bool, dict]
     """
     Set properties on a user record.
     This will overwrite previous people property values, just like `identify`.
@@ -119,24 +128,26 @@ def set(
      })
      ```
     """
-    _proxy(
+    return _proxy(
         "set",
         distinct_id=distinct_id,
         properties=properties,
         context=context,
         timestamp=timestamp,
         uuid=uuid,
+        disable_geoip=disable_geoip,
     )
 
 
 def set_once(
-    distinct_id,  # type: str,
+    distinct_id,  # type: str
     properties=None,  # type: Optional[Dict]
     context=None,  # type: Optional[Dict]
     timestamp=None,  # type: Optional[datetime.datetime]
     uuid=None,  # type: Optional[str]
+    disable_geoip=None,  # type: Optional[bool]
 ):
-    # type: (...) -> None
+    # type: (...) -> Tuple[bool, dict]
     """
     Set properties on a user record, only if they do not yet exist.
     This will not overwrite previous people property values, unlike `identify`.
@@ -152,13 +163,14 @@ def set_once(
      })
      ```
     """
-    _proxy(
+    return _proxy(
         "set_once",
         distinct_id=distinct_id,
         properties=properties,
         context=context,
         timestamp=timestamp,
         uuid=uuid,
+        disable_geoip=disable_geoip,
     )
 
 
@@ -169,8 +181,9 @@ def group_identify(
     context=None,  # type: Optional[Dict]
     timestamp=None,  # type: Optional[datetime.datetime]
     uuid=None,  # type: Optional[str]
+    disable_geoip=None,  # type: Optional[bool]
 ):
-    # type: (...) -> None
+    # type: (...) -> Tuple[bool, dict]
     """
     Set properties on a group
 
@@ -186,7 +199,7 @@ def group_identify(
      })
      ```
     """
-    _proxy(
+    return _proxy(
         "group_identify",
         group_type=group_type,
         group_key=group_key,
@@ -194,17 +207,19 @@ def group_identify(
         context=context,
         timestamp=timestamp,
         uuid=uuid,
+        disable_geoip=disable_geoip,
     )
 
 
 def alias(
-    previous_id,  # type: str,
-    distinct_id,  # type: str,
+    previous_id,  # type: str
+    distinct_id,  # type: str
     context=None,  # type: Optional[Dict]
     timestamp=None,  # type: Optional[datetime.datetime]
     uuid=None,  # type: Optional[str]
+    disable_geoip=None,  # type: Optional[bool]
 ):
-    # type: (...) -> None
+    # type: (...) -> Tuple[bool, dict]
     """
     To marry up whatever a user does before they sign up or log in with what they do after you need to make an alias call. This will allow you to answer questions like "Which marketing channels leads to users churning after a month?" or "What do users do on our website before signing up?"
 
@@ -221,21 +236,26 @@ def alias(
     posthog.alias('anonymous session id', 'distinct id')
     ```
     """
-    _proxy(
+    return _proxy(
         "alias",
         previous_id=previous_id,
         distinct_id=distinct_id,
         context=context,
         timestamp=timestamp,
         uuid=uuid,
+        disable_geoip=disable_geoip,
     )
 
 
 def feature_enabled(
-    key,  # type: str,
-    distinct_id,  # type: str,
-    default=False,  # type: bool
+    key,  # type: str
+    distinct_id,  # type: str
     groups={},  # type: dict
+    person_properties={},  # type: dict
+    group_properties={},  # type: dict
+    only_evaluate_locally=False,  # type: bool
+    send_feature_flag_events=True,  # type: bool
+    disable_geoip=None,  # type: Optional[bool]
 ):
     # type: (...) -> bool
     """
@@ -251,7 +271,144 @@ def feature_enabled(
 
     You can call `posthog.load_feature_flags()` before to make sure you're not doing unexpected requests.
     """
-    return _proxy("feature_enabled", key=key, distinct_id=distinct_id, default=default, groups=groups)
+    return _proxy(
+        "feature_enabled",
+        key=key,
+        distinct_id=distinct_id,
+        groups=groups,
+        person_properties=person_properties,
+        group_properties=group_properties,
+        only_evaluate_locally=only_evaluate_locally,
+        send_feature_flag_events=send_feature_flag_events,
+        disable_geoip=disable_geoip,
+    )
+
+
+def get_feature_flag(
+    key,  # type: str
+    distinct_id,  # type: str
+    groups={},  # type: dict
+    person_properties={},  # type: dict
+    group_properties={},  # type: dict
+    only_evaluate_locally=False,  # type: bool
+    send_feature_flag_events=True,  # type: bool
+    disable_geoip=None,  # type: Optional[bool]
+):
+    """
+    Get feature flag variant for users. Used with experiments.
+    Example:
+    ```python
+    if posthog.get_feature_flag('beta-feature', 'distinct_id') == 'test-variant':
+        # do test variant code
+    if posthog.get_feature_flag('beta-feature', 'distinct_id') == 'control':
+        # do control code
+    ```
+
+    `groups` are a mapping from group type to group key. So, if you have a group type of "organization" and a group key of "5",
+    you would pass groups={"organization": "5"}.
+
+    `group_properties` take the format: { group_type_name: { group_properties } }
+
+    So, for example, if you have the group type "organization" and the group key "5", with the properties name, and employee count,
+    you'll send these as:
+
+    ```python
+        group_properties={"organization": {"name": "PostHog", "employees": 11}}
+    ```
+    """
+    return _proxy(
+        "get_feature_flag",
+        key=key,
+        distinct_id=distinct_id,
+        groups=groups,
+        person_properties=person_properties,
+        group_properties=group_properties,
+        only_evaluate_locally=only_evaluate_locally,
+        send_feature_flag_events=send_feature_flag_events,
+        disable_geoip=disable_geoip,
+    )
+
+
+def get_all_flags(
+    distinct_id,  # type: str
+    groups={},  # type: dict
+    person_properties={},  # type: dict
+    group_properties={},  # type: dict
+    only_evaluate_locally=False,  # type: bool
+    disable_geoip=None,  # type: Optional[bool]
+):
+    """
+    Get all flags for a given user.
+    Example:
+    ```python
+    flags = posthog.get_all_flags('distinct_id')
+    ```
+
+    flags are key-value pairs where the key is the flag key and the value is the flag variant, or True, or False.
+    """
+    return _proxy(
+        "get_all_flags",
+        distinct_id=distinct_id,
+        groups=groups,
+        person_properties=person_properties,
+        group_properties=group_properties,
+        only_evaluate_locally=only_evaluate_locally,
+        disable_geoip=disable_geoip,
+    )
+
+
+def get_feature_flag_payload(
+    key,
+    distinct_id,
+    match_value=None,
+    groups={},
+    person_properties={},
+    group_properties={},
+    only_evaluate_locally=False,
+    send_feature_flag_events=True,
+    disable_geoip=None,  # type: Optional[bool]
+):
+    return _proxy(
+        "get_feature_flag_payload",
+        key=key,
+        distinct_id=distinct_id,
+        match_value=match_value,
+        groups=groups,
+        person_properties=person_properties,
+        group_properties=group_properties,
+        only_evaluate_locally=only_evaluate_locally,
+        send_feature_flag_events=send_feature_flag_events,
+        disable_geoip=disable_geoip,
+    )
+
+
+def get_all_flags_and_payloads(
+    distinct_id,
+    groups={},
+    person_properties={},
+    group_properties={},
+    only_evaluate_locally=False,
+    disable_geoip=None,  # type: Optional[bool]
+):
+    return _proxy(
+        "get_all_flags_and_payloads",
+        distinct_id=distinct_id,
+        groups=groups,
+        person_properties=person_properties,
+        group_properties=group_properties,
+        only_evaluate_locally=only_evaluate_locally,
+        disable_geoip=disable_geoip,
+    )
+
+
+def feature_flag_definitions():
+    """Returns loaded feature flags, if any. Helpful for debugging what flag information you have loaded."""
+    return _proxy("feature_flag_definitions")
+
+
+def load_feature_flags():
+    """Load feature flag definitions from PostHog."""
+    return _proxy("load_feature_flags")
 
 
 def page(*args, **kwargs):
@@ -283,8 +440,6 @@ def shutdown():
 def _proxy(method, *args, **kwargs):
     """Create an analytics client if one doesn't exist and send to it."""
     global default_client
-    if disabled:
-        return None
     if not default_client:
         default_client = Client(
             api_key,
@@ -295,7 +450,19 @@ def _proxy(method, *args, **kwargs):
             sync_mode=sync_mode,
             personal_api_key=personal_api_key,
             project_api_key=project_api_key,
+            poll_interval=poll_interval,
+            disabled=disabled,
+            disable_geoip=disable_geoip,
+            feature_flags_request_timeout_seconds=feature_flags_request_timeout_seconds,
         )
 
+    # always set incase user changes it
+    default_client.disabled = disabled
+    default_client.debug = debug
+
     fn = getattr(default_client, method)
     return fn(*args, **kwargs)
+
+
+class Posthog(Client):
+    pass

posthog/client.py

@@ -1,17 +1,17 @@
 import atexit
-import hashlib
 import logging
 import numbers
 from datetime import datetime, timedelta
-from uuid import UUID, uuid4
+from uuid import UUID
 
 from dateutil.tz import tzutc
 from six import string_types
 
 from posthog.consumer import Consumer
+from posthog.feature_flags import InconclusiveMatchError, match_feature_flag_properties
 from posthog.poller import Poller
-from posthog.request import APIError, batch_post, decide, get
-from posthog.utils import clean, guess_timezone
+from posthog.request import APIError, batch_post, decide, determine_server_host, get
+from posthog.utils import SizeLimitedDict, clean, guess_timezone
 from posthog.version import VERSION
 
 try:
@@ -21,7 +21,7 @@ except ImportError:
 
 
 ID_TYPES = (numbers.Number, string_types, UUID)
-__LONG_SCALE__ = float(0xFFFFFFFFFFFFFFF)
+MAX_DICT_SIZE = 50_000
 
 
 class Client(object):
@@ -47,8 +47,11 @@ class Client(object):
         poll_interval=30,
         personal_api_key=None,
         project_api_key=None,
+        disabled=False,
+        disable_geoip=True,
+        historical_migration=False,
+        feature_flags_request_timeout_seconds=3,
     ):
-
         self.queue = queue.Queue(max_queue_size)
 
         # api_key: This should be the Team API Key (token), public
@@ -60,18 +63,30 @@ class Client(object):
         self.debug = debug
         self.send = send
         self.sync_mode = sync_mode
-        self.host = host
+        self.host = determine_server_host(host)
         self.gzip = gzip
         self.timeout = timeout
         self.feature_flags = None
+        self.feature_flags_by_key = None
+        self.group_type_mapping = None
+        self.cohorts = None
         self.poll_interval = poll_interval
+        self.feature_flags_request_timeout_seconds = feature_flags_request_timeout_seconds
         self.poller = None
+        self.distinct_ids_feature_flags_reported = SizeLimitedDict(MAX_DICT_SIZE, set)
+        self.disabled = disabled
+        self.disable_geoip = disable_geoip
+        self.historical_migration = historical_migration
 
         # personal_api_key: This should be a generated Personal API Key, private
         self.personal_api_key = personal_api_key
-
         if debug:
+            # Ensures that debug level messages are logged when debug mode is on.
+            # Otherwise, defaults to WARNING level. See https://docs.python.org/3/howto/logging.html#what-happens-if-no-configuration-is-provided
+            logging.basicConfig()
             self.log.setLevel(logging.DEBUG)
+        else:
+            self.log.setLevel(logging.WARNING)
 
         if sync_mode:
             self.consumers = None
@@ -89,13 +104,14 @@ class Client(object):
                 consumer = Consumer(
                     self.queue,
                     self.api_key,
-                    host=host,
+                    host=self.host,
                     on_error=on_error,
                     flush_at=flush_at,
                     flush_interval=flush_interval,
                     gzip=gzip,
                     retries=max_retries,
                     timeout=timeout,
+                    historical_migration=historical_migration,
                 )
                 self.consumers.append(consumer)
 
@@ -103,7 +119,7 @@ class Client(object):
                 if send:
                     consumer.start()
 
-    def identify(self, distinct_id=None, properties=None, context=None, timestamp=None, uuid=None):
+    def identify(self, distinct_id=None, properties=None, context=None, timestamp=None, uuid=None, disable_geoip=None):
         properties = properties or {}
         context = context or {}
         require("distinct_id", distinct_id, ID_TYPES)
@@ -118,12 +134,26 @@ class Client(object):
             "uuid": uuid,
         }
 
-        return self._enqueue(msg)
+        return self._enqueue(msg, disable_geoip)
 
-    def get_feature_variants(self, distinct_id, groups=None):
-        assert self.personal_api_key, "You have to specify a personal_api_key to use feature flags."
+    def get_feature_variants(
+        self, distinct_id, groups=None, person_properties=None, group_properties=None, disable_geoip=None
+    ):
+        resp_data = self.get_decide(distinct_id, groups, person_properties, group_properties, disable_geoip)
+        return resp_data["featureFlags"]
+
+    def get_feature_payloads(
+        self, distinct_id, groups=None, person_properties=None, group_properties=None, disable_geoip=None
+    ):
+        resp_data = self.get_decide(distinct_id, groups, person_properties, group_properties, disable_geoip)
+        return resp_data["featureFlagPayloads"]
+
+    def get_decide(self, distinct_id, groups=None, person_properties=None, group_properties=None, disable_geoip=None):
         require("distinct_id", distinct_id, ID_TYPES)
 
+        if disable_geoip is None:
+            disable_geoip = self.disable_geoip
+
         if groups:
             require("groups", groups, dict)
         else:
@@ -131,11 +161,14 @@ class Client(object):
 
         request_data = {
             "distinct_id": distinct_id,
-            "personal_api_key": self.personal_api_key,
             "groups": groups,
+            "person_properties": person_properties,
+            "group_properties": group_properties,
+            "disable_geoip": disable_geoip,
         }
-        resp_data = decide(self.api_key, self.host, timeout=10, **request_data)
-        return resp_data["featureFlags"]
+        resp_data = decide(self.api_key, self.host, timeout=self.feature_flags_request_timeout_seconds, **request_data)
+
+        return resp_data
 
     def capture(
         self,
@@ -147,6 +180,7 @@ class Client(object):
         uuid=None,
         groups=None,
         send_feature_flags=False,
+        disable_geoip=None,
     ):
         properties = properties or {}
         context = context or {}
@@ -167,19 +201,33 @@ class Client(object):
             require("groups", groups, dict)
             msg["properties"]["$groups"] = groups
 
+        extra_properties = {}
+        feature_variants = {}
         if send_feature_flags:
             try:
-                feature_variants = self.get_feature_variants(distinct_id, groups)
+                feature_variants = self.get_feature_variants(distinct_id, groups, disable_geoip=disable_geoip)
             except Exception as e:
                 self.log.exception(f"[FEATURE FLAGS] Unable to get feature variants: {e}")
-            else:
-                for feature, variant in feature_variants.items():
-                    msg["properties"]["$feature/{}".format(feature)] = variant
-                msg["properties"]["$active_feature_flags"] = list(feature_variants.keys())
 
-        return self._enqueue(msg)
+        elif self.feature_flags:
+            # Local evaluation is enabled, flags are loaded, so try and get all flags we can without going to the server
+            feature_variants = self.get_all_flags(
+                distinct_id, groups=(groups or {}), disable_geoip=disable_geoip, only_evaluate_locally=True
+            )
 
-    def set(self, distinct_id=None, properties=None, context=None, timestamp=None, uuid=None):
+        for feature, variant in feature_variants.items():
+            extra_properties[f"$feature/{feature}"] = variant
+
+        active_feature_flags = [key for (key, value) in feature_variants.items() if value is not False]
+        if active_feature_flags:
+            extra_properties["$active_feature_flags"] = active_feature_flags
+
+        if extra_properties:
+            msg["properties"] = {**extra_properties, **msg["properties"]}
+
+        return self._enqueue(msg, disable_geoip)
+
+    def set(self, distinct_id=None, properties=None, context=None, timestamp=None, uuid=None, disable_geoip=None):
         properties = properties or {}
         context = context or {}
         require("distinct_id", distinct_id, ID_TYPES)
@@ -194,9 +242,9 @@ class Client(object):
             "uuid": uuid,
         }
 
-        return self._enqueue(msg)
+        return self._enqueue(msg, disable_geoip)
 
-    def set_once(self, distinct_id=None, properties=None, context=None, timestamp=None, uuid=None):
+    def set_once(self, distinct_id=None, properties=None, context=None, timestamp=None, uuid=None, disable_geoip=None):
         properties = properties or {}
         context = context or {}
         require("distinct_id", distinct_id, ID_TYPES)
@@ -211,9 +259,18 @@ class Client(object):
             "uuid": uuid,
         }
 
-        return self._enqueue(msg)
+        return self._enqueue(msg, disable_geoip)
 
-    def group_identify(self, group_type=None, group_key=None, properties=None, context=None, timestamp=None, uuid=None):
+    def group_identify(
+        self,
+        group_type=None,
+        group_key=None,
+        properties=None,
+        context=None,
+        timestamp=None,
+        uuid=None,
+        disable_geoip=None,
+    ):
         properties = properties or {}
         context = context or {}
         require("group_type", group_type, ID_TYPES)
@@ -233,9 +290,9 @@ class Client(object):
             "uuid": uuid,
         }
 
-        return self._enqueue(msg)
+        return self._enqueue(msg, disable_geoip)
 
-    def alias(self, previous_id=None, distinct_id=None, context=None, timestamp=None, uuid=None):
+    def alias(self, previous_id=None, distinct_id=None, context=None, timestamp=None, uuid=None, disable_geoip=None):
         context = context or {}
 
         require("previous_id", previous_id, ID_TYPES)
@@ -252,9 +309,11 @@ class Client(object):
             "distinct_id": previous_id,
         }
 
-        return self._enqueue(msg)
+        return self._enqueue(msg, disable_geoip)
 
-    def page(self, distinct_id=None, url=None, properties=None, context=None, timestamp=None, uuid=None):
+    def page(
+        self, distinct_id=None, url=None, properties=None, context=None, timestamp=None, uuid=None, disable_geoip=None
+    ):
         properties = properties or {}
         context = context or {}
 
@@ -273,10 +332,14 @@ class Client(object):
             "uuid": uuid,
         }
 
-        return self._enqueue(msg)
+        return self._enqueue(msg, disable_geoip)
 
-    def _enqueue(self, msg):
+    def _enqueue(self, msg, disable_geoip):
         """Push a new `msg` onto the queue, return `(success, msg)`"""
+
+        if self.disabled:
+            return False, "disabled"
+
         timestamp = msg["timestamp"]
         if timestamp is None:
             timestamp = datetime.utcnow().replace(tzinfo=tzutc())
@@ -299,6 +362,12 @@ class Client(object):
         msg["properties"]["$lib"] = "posthog-python"
         msg["properties"]["$lib_version"] = VERSION
 
+        if disable_geoip is None:
+            disable_geoip = self.disable_geoip
+
+        if disable_geoip:
+            msg["properties"]["$geoip_disable"] = True
+
         msg["distinct_id"] = stringify_id(msg.get("distinct_id", None))
 
         msg = clean(msg)
@@ -310,7 +379,14 @@ class Client(object):
 
         if self.sync_mode:
             self.log.debug("enqueued with blocking %s.", msg["event"])
-            batch_post(self.api_key, self.host, gzip=self.gzip, timeout=self.timeout, batch=[msg])
+            batch_post(
+                self.api_key,
+                self.host,
+                gzip=self.gzip,
+                timeout=self.timeout,
+                batch=[msg],
+                historical_migration=self.historical_migration,
+            )
 
             return True, msg
 
@@ -352,18 +428,34 @@ class Client(object):
 
     def _load_feature_flags(self):
         try:
-            flags = get(self.personal_api_key, f"/api/feature_flag/?token={self.api_key}", self.host)["results"]
-            self.feature_flags = [flag for flag in flags if flag["active"]]
+            response = get(
+                self.personal_api_key,
+                f"/api/feature_flag/local_evaluation/?token={self.api_key}&send_cohorts",
+                self.host,
+                timeout=10,
+            )
+
+            self.feature_flags = response["flags"] or []
+            self.feature_flags_by_key = {
+                flag["key"]: flag for flag in self.feature_flags if flag.get("key") is not None
+            }
+            self.group_type_mapping = response["group_type_mapping"] or {}
+            self.cohorts = response["cohorts"] or {}
+
         except APIError as e:
             if e.status == 401:
-                raise APIError(
-                    status=401,
-                    message="You are using a write-only key with feature flags. "
-                    "To use feature flags, please set a personal_api_key "
-                    "More information: https://posthog.com/docs/api/overview",
+                self.log.error(
+                    "[FEATURE FLAGS] Error loading feature flags: To use feature flags, please set a valid personal_api_key. More information: https://posthog.com/docs/api/overview"
                 )
+                if self.debug:
+                    raise APIError(
+                        status=401,
+                        message="You are using a write-only key with feature flags. "
+                        "To use feature flags, please set a personal_api_key "
+                        "More information: https://posthog.com/docs/api/overview",
+                    )
             else:
-                raise APIError(status=e.status, message=e.message)
+                self.log.error(f"[FEATURE FLAGS] Error loading feature flags: {e}")
         except Exception as e:
             self.log.warning(
                 "[FEATURE FLAGS] Fetching feature flags failed with following error. We will retry in %s seconds."
@@ -384,47 +476,324 @@ class Client(object):
             self.poller = Poller(interval=timedelta(seconds=self.poll_interval), execute=self._load_feature_flags)
             self.poller.start()
 
-    def feature_enabled(self, key, distinct_id, default=False, *, groups={}):
+    def _compute_flag_locally(
+        self,
+        feature_flag,
+        distinct_id,
+        *,
+        groups={},
+        person_properties={},
+        group_properties={},
+        warn_on_unknown_groups=True,
+    ):
+        if feature_flag.get("ensure_experience_continuity", False):
+            raise InconclusiveMatchError("Flag has experience continuity enabled")
+
+        if not feature_flag.get("active"):
+            return False
+
+        flag_filters = feature_flag.get("filters") or {}
+        aggregation_group_type_index = flag_filters.get("aggregation_group_type_index")
+        if aggregation_group_type_index is not None:
+            group_name = self.group_type_mapping.get(str(aggregation_group_type_index))
+
+            if not group_name:
+                self.log.warning(
+                    f"[FEATURE FLAGS] Unknown group type index {aggregation_group_type_index} for feature flag {feature_flag['key']}"
+                )
+                # failover to `/decide/`
+                raise InconclusiveMatchError("Flag has unknown group type index")
+
+            if group_name not in groups:
+                # Group flags are never enabled in `groups` aren't passed in
+                # don't failover to `/decide/`, since response will be the same
+                if warn_on_unknown_groups:
+                    self.log.warning(
+                        f"[FEATURE FLAGS] Can't compute group feature flag: {feature_flag['key']} without group names passed in"
+                    )
+                else:
+                    self.log.debug(
+                        f"[FEATURE FLAGS] Can't compute group feature flag: {feature_flag['key']} without group names passed in"
+                    )
+                return False
+
+            focused_group_properties = group_properties[group_name]
+            return match_feature_flag_properties(feature_flag, groups[group_name], focused_group_properties)
+        else:
+            return match_feature_flag_properties(feature_flag, distinct_id, person_properties, self.cohorts)
+
+    def feature_enabled(
+        self,
+        key,
+        distinct_id,
+        *,
+        groups={},
+        person_properties={},
+        group_properties={},
+        only_evaluate_locally=False,
+        send_feature_flag_events=True,
+        disable_geoip=None,
+    ):
+        response = self.get_feature_flag(
+            key,
+            distinct_id,
+            groups=groups,
+            person_properties=person_properties,
+            group_properties=group_properties,
+            only_evaluate_locally=only_evaluate_locally,
+            send_feature_flag_events=send_feature_flag_events,
+            disable_geoip=disable_geoip,
+        )
+
+        if response is None:
+            return None
+        return bool(response)
+
+    def get_feature_flag(
+        self,
+        key,
+        distinct_id,
+        *,
+        groups={},
+        person_properties={},
+        group_properties={},
+        only_evaluate_locally=False,
+        send_feature_flag_events=True,
+        disable_geoip=None,
+    ):
         require("key", key, string_types)
         require("distinct_id", distinct_id, ID_TYPES)
         require("groups", groups, dict)
 
-        if self.feature_flags == None and self.personal_api_key:
+        if self.disabled:
+            return None
+
+        person_properties, group_properties = self._add_local_person_and_group_properties(
+            distinct_id, groups, person_properties, group_properties
+        )
+
+        if self.feature_flags is None and self.personal_api_key:
             self.load_feature_flags()
         response = None
 
         # If loading in previous line failed
         if self.feature_flags:
-            for flag in self.feature_flags:
-                if flag["key"] == key:
-                    feature_flag = flag
-                    if feature_flag.get("is_simple_flag"):
-                        rollout_percentage = (
-                            feature_flag.get("rollout_percentage")
-                            if feature_flag.get("rollout_percentage") is not None
-                            else 100
-                        )
-                        response = _hash(key, distinct_id) <= (rollout_percentage / 100)
-        if response == None:
+            flag = self.feature_flags_by_key.get(key)
+            if flag:
+                try:
+                    response = self._compute_flag_locally(
+                        flag,
+                        distinct_id,
+                        groups=groups,
+                        person_properties=person_properties,
+                        group_properties=group_properties,
+                    )
+                    self.log.debug(f"Successfully computed flag locally: {key} -> {response}")
+                except InconclusiveMatchError as e:
+                    self.log.debug(f"Failed to compute flag {key} locally: {e}")
+                except Exception as e:
+                    self.log.exception(f"[FEATURE FLAGS] Error while computing variant locally: {e}")
+
+        flag_was_locally_evaluated = response is not None
+        if not flag_was_locally_evaluated and not only_evaluate_locally:
             try:
-                feature_flags = self.get_feature_variants(distinct_id, groups=groups)
+                feature_flags = self.get_feature_variants(
+                    distinct_id,
+                    groups=groups,
+                    person_properties=person_properties,
+                    group_properties=group_properties,
+                    disable_geoip=disable_geoip,
+                )
+                response = feature_flags.get(key)
+                if response is None:
+                    response = False
+                self.log.debug(f"Successfully computed flag remotely: #{key} -> #{response}")
             except Exception as e:
-                self.log.exception(f"[FEATURE FLAGS] Unable to get feature variants: {e}")
-                response = default
-            else:
-                response = feature_flags.get(key, default)
-        self.capture(distinct_id, "$feature_flag_called", {"$feature_flag": key, "$feature_flag_response": response})
+                self.log.exception(f"[FEATURE FLAGS] Unable to get flag remotely: {e}")
+
+        feature_flag_reported_key = f"{key}_{str(response)}"
+        if (
+            feature_flag_reported_key not in self.distinct_ids_feature_flags_reported[distinct_id]
+            and send_feature_flag_events  # noqa: W503
+        ):
+            self.capture(
+                distinct_id,
+                "$feature_flag_called",
+                {
+                    "$feature_flag": key,
+                    "$feature_flag_response": response,
+                    "locally_evaluated": flag_was_locally_evaluated,
+                    f"$feature/{key}": response,
+                },
+                groups=groups,
+                disable_geoip=disable_geoip,
+            )
+            self.distinct_ids_feature_flags_reported[distinct_id].add(feature_flag_reported_key)
         return response
 
+    def get_feature_flag_payload(
+        self,
+        key,
+        distinct_id,
+        *,
+        match_value=None,
+        groups={},
+        person_properties={},
+        group_properties={},
+        only_evaluate_locally=False,
+        send_feature_flag_events=True,
+        disable_geoip=None,
+    ):
+        if self.disabled:
+            return None
 
-# This function takes a distinct_id and a feature flag key and returns a float between 0 and 1.
-# Given the same distinct_id and key, it'll always return the same float. These floats are
-# uniformly distributed between 0 and 1, so if we want to show this feature to 20% of traffic
-# we can do _hash(key, distinct_id) < 0.2
-def _hash(key, distinct_id):
-    hash_key = "%s.%s" % (key, distinct_id)
-    hash_val = int(hashlib.sha1(hash_key.encode("utf-8")).hexdigest()[:15], 16)
-    return hash_val / __LONG_SCALE__
+        if match_value is None:
+            match_value = self.get_feature_flag(
+                key,
+                distinct_id,
+                groups=groups,
+                person_properties=person_properties,
+                group_properties=group_properties,
+                send_feature_flag_events=send_feature_flag_events,
+                only_evaluate_locally=True,
+                disable_geoip=disable_geoip,
+            )
+
+        response = None
+
+        if match_value is not None:
+            response = self._compute_payload_locally(key, match_value)
+
+        if response is None and not only_evaluate_locally:
+            decide_payloads = self.get_feature_payloads(
+                distinct_id, groups, person_properties, group_properties, disable_geoip
+            )
+            response = decide_payloads.get(str(key).lower(), None)
+
+        return response
+
+    def _compute_payload_locally(self, key, match_value):
+        payload = None
+
+        if self.feature_flags_by_key is None:
+            return payload
+
+        flag_definition = self.feature_flags_by_key.get(key) or {}
+        flag_filters = flag_definition.get("filters") or {}
+        flag_payloads = flag_filters.get("payloads") or {}
+        payload = flag_payloads.get(str(match_value).lower(), None)
+        return payload
+
+    def get_all_flags(
+        self,
+        distinct_id,
+        *,
+        groups={},
+        person_properties={},
+        group_properties={},
+        only_evaluate_locally=False,
+        disable_geoip=None,
+    ):
+        flags = self.get_all_flags_and_payloads(
+            distinct_id,
+            groups=groups,
+            person_properties=person_properties,
+            group_properties=group_properties,
+            only_evaluate_locally=only_evaluate_locally,
+            disable_geoip=disable_geoip,
+        )
+        return flags["featureFlags"]
+
+    def get_all_flags_and_payloads(
+        self,
+        distinct_id,
+        *,
+        groups={},
+        person_properties={},
+        group_properties={},
+        only_evaluate_locally=False,
+        disable_geoip=None,
+    ):
+        if self.disabled:
+            return {"featureFlags": None, "featureFlagPayloads": None}
+
+        person_properties, group_properties = self._add_local_person_and_group_properties(
+            distinct_id, groups, person_properties, group_properties
+        )
+
+        flags, payloads, fallback_to_decide = self._get_all_flags_and_payloads_locally(
+            distinct_id, groups=groups, person_properties=person_properties, group_properties=group_properties
+        )
+        response = {"featureFlags": flags, "featureFlagPayloads": payloads}
+
+        if fallback_to_decide and not only_evaluate_locally:
+            try:
+                flags_and_payloads = self.get_decide(
+                    distinct_id,
+                    groups=groups,
+                    person_properties=person_properties,
+                    group_properties=group_properties,
+                    disable_geoip=disable_geoip,
+                )
+                response = flags_and_payloads
+            except Exception as e:
+                self.log.exception(f"[FEATURE FLAGS] Unable to get feature flags and payloads: {e}")
+
+        return response
+
+    def _get_all_flags_and_payloads_locally(
+        self, distinct_id, *, groups={}, person_properties={}, group_properties={}, warn_on_unknown_groups=False
+    ):
+        require("distinct_id", distinct_id, ID_TYPES)
+        require("groups", groups, dict)
+
+        if self.feature_flags is None and self.personal_api_key:
+            self.load_feature_flags()
+
+        flags = {}
+        payloads = {}
+        fallback_to_decide = False
+        # If loading in previous line failed
+        if self.feature_flags:
+            for flag in self.feature_flags:
+                try:
+                    flags[flag["key"]] = self._compute_flag_locally(
+                        flag,
+                        distinct_id,
+                        groups=groups,
+                        person_properties=person_properties,
+                        group_properties=group_properties,
+                        warn_on_unknown_groups=warn_on_unknown_groups,
+                    )
+                    matched_payload = self._compute_payload_locally(flag["key"], flags[flag["key"]])
+                    if matched_payload:
+                        payloads[flag["key"]] = matched_payload
+                except InconclusiveMatchError:
+                    # No need to log this, since it's just telling us to fall back to `/decide`
+                    fallback_to_decide = True
+                except Exception as e:
+                    self.log.exception(f"[FEATURE FLAGS] Error while computing variant and payload: {e}")
+                    fallback_to_decide = True
+        else:
+            fallback_to_decide = True
+
+        return flags, payloads, fallback_to_decide
+
+    def feature_flag_definitions(self):
+        return self.feature_flags
+
+    def _add_local_person_and_group_properties(self, distinct_id, groups, person_properties, group_properties):
+        all_person_properties = {"distinct_id": distinct_id, **(person_properties or {})}
+
+        all_group_properties = {}
+        if groups:
+            for group_name in groups:
+                all_group_properties[group_name] = {
+                    "$group_key": groups[group_name],
+                    **(group_properties.get(group_name) or {}),
+                }
+
+        return all_person_properties, all_group_properties
 
 
 def require(name, field, data_type):

posthog/consumer.py

@@ -12,11 +12,12 @@ try:
 except ImportError:
     from Queue import Empty
 
-MAX_MSG_SIZE = 32 << 10
 
-# Our servers only accept batches less than 500KB. Here limit is set slightly
-# lower to leave space for extra data that will be added later, eg. "sentAt".
-BATCH_SIZE_LIMIT = 475000
+MAX_MSG_SIZE = 900 * 1024  # 900KiB per event
+
+# The maximum request body size is currently 20MiB, let's be conservative
+# in case we want to lower it in the future.
+BATCH_SIZE_LIMIT = 5 * 1024 * 1024
 
 
 class Consumer(Thread):
@@ -35,6 +36,7 @@ class Consumer(Thread):
         gzip=False,
         retries=10,
         timeout=15,
+        historical_migration=False,
     ):
         """Create a consumer thread."""
         Thread.__init__(self)
@@ -54,6 +56,7 @@ class Consumer(Thread):
         self.running = True
         self.retries = retries
         self.timeout = timeout
+        self.historical_migration = historical_migration
 
     def run(self):
         """Runs the consumer."""
@@ -104,7 +107,7 @@ class Consumer(Thread):
                 item = queue.get(block=True, timeout=self.flush_interval - elapsed)
                 item_size = len(json.dumps(item, cls=DatetimeSerializer).encode())
                 if item_size > MAX_MSG_SIZE:
-                    self.log.error("Item exceeds 32kb limit, dropping. (%s)", str(item))
+                    self.log.error("Item exceeds 900kib limit, dropping. (%s)", str(item))
                     continue
                 items.append(item)
                 total_size += item_size
@@ -133,6 +136,13 @@ class Consumer(Thread):
 
         @backoff.on_exception(backoff.expo, Exception, max_tries=self.retries + 1, giveup=fatal_exception)
         def send_request():
-            batch_post(self.api_key, self.host, gzip=self.gzip, timeout=self.timeout, batch=batch)
+            batch_post(
+                self.api_key,
+                self.host,
+                gzip=self.gzip,
+                timeout=self.timeout,
+                batch=batch,
+                historical_migration=self.historical_migration,
+            )
 
         send_request()

posthog/feature_flags.py

@@ -0,0 +1,331 @@
+import datetime
+import hashlib
+import logging
+import re
+from typing import Optional
+
+from dateutil import parser
+from dateutil.relativedelta import relativedelta
+
+from posthog.utils import convert_to_datetime_aware, is_valid_regex
+
+__LONG_SCALE__ = float(0xFFFFFFFFFFFFFFF)
+
+log = logging.getLogger("posthog")
+
+
+class InconclusiveMatchError(Exception):
+    pass
+
+
+# This function takes a distinct_id and a feature flag key and returns a float between 0 and 1.
+# Given the same distinct_id and key, it'll always return the same float. These floats are
+# uniformly distributed between 0 and 1, so if we want to show this feature to 20% of traffic
+# we can do _hash(key, distinct_id) < 0.2
+def _hash(key, distinct_id, salt=""):
+    hash_key = f"{key}.{distinct_id}{salt}"
+    hash_val = int(hashlib.sha1(hash_key.encode("utf-8")).hexdigest()[:15], 16)
+    return hash_val / __LONG_SCALE__
+
+
+def get_matching_variant(flag, distinct_id):
+    hash_value = _hash(flag["key"], distinct_id, salt="variant")
+    for variant in variant_lookup_table(flag):
+        if hash_value >= variant["value_min"] and hash_value < variant["value_max"]:
+            return variant["key"]
+    return None
+
+
+def variant_lookup_table(feature_flag):
+    lookup_table = []
+    value_min = 0
+    multivariates = ((feature_flag.get("filters") or {}).get("multivariate") or {}).get("variants") or []
+    for variant in multivariates:
+        value_max = value_min + variant["rollout_percentage"] / 100
+        lookup_table.append({"value_min": value_min, "value_max": value_max, "key": variant["key"]})
+        value_min = value_max
+    return lookup_table
+
+
+def match_feature_flag_properties(flag, distinct_id, properties, cohort_properties=None):
+    flag_conditions = (flag.get("filters") or {}).get("groups") or []
+    is_inconclusive = False
+    cohort_properties = cohort_properties or {}
+
+    # Stable sort conditions with variant overrides to the top. This ensures that if overrides are present, they are
+    # evaluated first, and the variant override is applied to the first matching condition.
+    sorted_flag_conditions = sorted(
+        flag_conditions,
+        key=lambda condition: 0 if condition.get("variant") else 1,
+    )
+
+    for condition in sorted_flag_conditions:
+        try:
+            # if any one condition resolves to True, we can shortcircuit and return
+            # the matching variant
+            if is_condition_match(flag, distinct_id, condition, properties, cohort_properties):
+                variant_override = condition.get("variant")
+                # Some filters can be explicitly set to null, which require accessing variants like so
+                flag_variants = ((flag.get("filters") or {}).get("multivariate") or {}).get("variants") or []
+                if variant_override and variant_override in [variant["key"] for variant in flag_variants]:
+                    variant = variant_override
+                else:
+                    variant = get_matching_variant(flag, distinct_id)
+                return variant or True
+        except InconclusiveMatchError:
+            is_inconclusive = True
+
+    if is_inconclusive:
+        raise InconclusiveMatchError("Can't determine if feature flag is enabled or not with given properties")
+
+    # We can only return False when either all conditions are False, or
+    # no condition was inconclusive.
+    return False
+
+
+def is_condition_match(feature_flag, distinct_id, condition, properties, cohort_properties):
+    rollout_percentage = condition.get("rollout_percentage")
+    if len(condition.get("properties") or []) > 0:
+        for prop in condition.get("properties"):
+            property_type = prop.get("type")
+            if property_type == "cohort":
+                matches = match_cohort(prop, properties, cohort_properties)
+            else:
+                matches = match_property(prop, properties)
+            if not matches:
+                return False
+
+        if rollout_percentage is None:
+            return True
+
+    if rollout_percentage is not None and _hash(feature_flag["key"], distinct_id) > (rollout_percentage / 100):
+        return False
+
+    return True
+
+
+def match_property(property, property_values) -> bool:
+    # only looks for matches where key exists in override_property_values
+    # doesn't support operator is_not_set
+    key = property.get("key")
+    operator = property.get("operator") or "exact"
+    value = property.get("value")
+
+    if key not in property_values:
+        raise InconclusiveMatchError("can't match properties without a given property value")
+
+    if operator == "is_not_set":
+        raise InconclusiveMatchError("can't match properties with operator is_not_set")
+
+    override_value = property_values[key]
+
+    if operator in ("exact", "is_not"):
+
+        def compute_exact_match(value, override_value):
+            if isinstance(value, list):
+                return str(override_value).lower() in [str(val).lower() for val in value]
+            return str(value).lower() == str(override_value).lower()
+
+        if operator == "exact":
+            return compute_exact_match(value, override_value)
+        else:
+            return not compute_exact_match(value, override_value)
+
+    if operator == "is_set":
+        return key in property_values
+
+    if operator == "icontains":
+        return str(value).lower() in str(override_value).lower()
+
+    if operator == "not_icontains":
+        return str(value).lower() not in str(override_value).lower()
+
+    if operator == "regex":
+        return is_valid_regex(str(value)) and re.compile(str(value)).search(str(override_value)) is not None
+
+    if operator == "not_regex":
+        return is_valid_regex(str(value)) and re.compile(str(value)).search(str(override_value)) is None
+
+    if operator in ("gt", "gte", "lt", "lte"):
+        # :TRICKY: We adjust comparison based on the override value passed in,
+        # to make sure we handle both numeric and string comparisons appropriately.
+        def compare(lhs, rhs, operator):
+            if operator == "gt":
+                return lhs > rhs
+            elif operator == "gte":
+                return lhs >= rhs
+            elif operator == "lt":
+                return lhs < rhs
+            elif operator == "lte":
+                return lhs <= rhs
+            else:
+                raise ValueError(f"Invalid operator: {operator}")
+
+        parsed_value = None
+        try:
+            parsed_value = float(value)  # type: ignore
+        except Exception:
+            pass
+
+        if parsed_value is not None and override_value is not None:
+            if isinstance(override_value, str):
+                return compare(override_value, str(value), operator)
+            else:
+                return compare(override_value, parsed_value, operator)
+        else:
+            return compare(str(override_value), str(value), operator)
+
+    if operator in ["is_date_before", "is_date_after"]:
+        try:
+            parsed_date = relative_date_parse_for_feature_flag_matching(str(value))
+
+            if not parsed_date:
+                parsed_date = parser.parse(str(value))
+                parsed_date = convert_to_datetime_aware(parsed_date)
+        except Exception as e:
+            raise InconclusiveMatchError("The date set on the flag is not a valid format") from e
+
+        if not parsed_date:
+            raise InconclusiveMatchError("The date set on the flag is not a valid format")
+
+        if isinstance(override_value, datetime.datetime):
+            override_date = convert_to_datetime_aware(override_value)
+            if operator == "is_date_before":
+                return override_date < parsed_date
+            else:
+                return override_date > parsed_date
+        elif isinstance(override_value, datetime.date):
+            if operator == "is_date_before":
+                return override_value < parsed_date.date()
+            else:
+                return override_value > parsed_date.date()
+        elif isinstance(override_value, str):
+            try:
+                override_date = parser.parse(override_value)
+                override_date = convert_to_datetime_aware(override_date)
+                if operator == "is_date_before":
+                    return override_date < parsed_date
+                else:
+                    return override_date > parsed_date
+            except Exception:
+                raise InconclusiveMatchError("The date provided is not a valid format")
+        else:
+            raise InconclusiveMatchError("The date provided must be a string or date object")
+
+    # if we get here, we don't know how to handle the operator
+    raise InconclusiveMatchError(f"Unknown operator {operator}")
+
+
+def match_cohort(property, property_values, cohort_properties) -> bool:
+    # Cohort properties are in the form of property groups like this:
+    # {
+    #     "cohort_id": {
+    #         "type": "AND|OR",
+    #         "values": [{
+    #            "key": "property_name", "value": "property_value"
+    #        }]
+    #     }
+    # }
+    cohort_id = str(property.get("value"))
+    if cohort_id not in cohort_properties:
+        raise InconclusiveMatchError("can't match cohort without a given cohort property value")
+
+    property_group = cohort_properties[cohort_id]
+    return match_property_group(property_group, property_values, cohort_properties)
+
+
+def match_property_group(property_group, property_values, cohort_properties) -> bool:
+    if not property_group:
+        return True
+
+    property_group_type = property_group.get("type")
+    properties = property_group.get("values")
+
+    if not properties or len(properties) == 0:
+        # empty groups are no-ops, always match
+        return True
+
+    error_matching_locally = False
+
+    if "values" in properties[0]:
+        # a nested property group
+        for prop in properties:
+            try:
+                matches = match_property_group(prop, property_values, cohort_properties)
+                if property_group_type == "AND":
+                    if not matches:
+                        return False
+                else:
+                    # OR group
+                    if matches:
+                        return True
+            except InconclusiveMatchError as e:
+                log.debug(f"Failed to compute property {prop} locally: {e}")
+                error_matching_locally = True
+
+        if error_matching_locally:
+            raise InconclusiveMatchError("Can't match cohort without a given cohort property value")
+        # if we get here, all matched in AND case, or none matched in OR case
+        return property_group_type == "AND"
+
+    else:
+        for prop in properties:
+            try:
+                if prop.get("type") == "cohort":
+                    matches = match_cohort(prop, property_values, cohort_properties)
+                else:
+                    matches = match_property(prop, property_values)
+
+                negation = prop.get("negation", False)
+
+                if property_group_type == "AND":
+                    # if negated property, do the inverse
+                    if not matches and not negation:
+                        return False
+                    if matches and negation:
+                        return False
+                else:
+                    # OR group
+                    if matches and not negation:
+                        return True
+                    if not matches and negation:
+                        return True
+            except InconclusiveMatchError as e:
+                log.debug(f"Failed to compute property {prop} locally: {e}")
+                error_matching_locally = True
+
+        if error_matching_locally:
+            raise InconclusiveMatchError("can't match cohort without a given cohort property value")
+
+        # if we get here, all matched in AND case, or none matched in OR case
+        return property_group_type == "AND"
+
+
+def relative_date_parse_for_feature_flag_matching(value: str) -> Optional[datetime.datetime]:
+    regex = r"^-?(?P<number>[0-9]+)(?P<interval>[a-z])$"
+    match = re.search(regex, value)
+    parsed_dt = datetime.datetime.now(datetime.timezone.utc)
+    if match:
+        number = int(match.group("number"))
+
+        if number >= 10_000:
+            # Guard against overflow, disallow numbers greater than 10_000
+            return None
+
+        interval = match.group("interval")
+        if interval == "h":
+            parsed_dt = parsed_dt - relativedelta(hours=number)
+        elif interval == "d":
+            parsed_dt = parsed_dt - relativedelta(days=number)
+        elif interval == "w":
+            parsed_dt = parsed_dt - relativedelta(weeks=number)
+        elif interval == "m":
+            parsed_dt = parsed_dt - relativedelta(months=number)
+        elif interval == "y":
+            parsed_dt = parsed_dt - relativedelta(years=number)
+        else:
+            return None
+
+        return parsed_dt
+    else:
+        return None

posthog/request.py

@@ -13,10 +13,24 @@ from posthog.version import VERSION
 
 _session = requests.sessions.Session()
 
-DEFAULT_HOST = "https://app.posthog.com"
+US_INGESTION_ENDPOINT = "https://us-api.i.posthog.com"
+EU_INGESTION_ENDPOINT = "https://eu-api.i.posthog.com"
+DEFAULT_HOST = US_INGESTION_ENDPOINT
 USER_AGENT = "posthog-python/" + VERSION
 
 
+def determine_server_host(host: Optional[str]) -> str:
+    """Determines the server host to use."""
+    host_or_default = host or DEFAULT_HOST
+    trimmed_host = remove_trailing_slash(host_or_default)
+    if trimmed_host in ("https://app.posthog.com", "https://us.posthog.com"):
+        return US_INGESTION_ENDPOINT
+    elif trimmed_host == "https://eu.posthog.com":
+        return EU_INGESTION_ENDPOINT
+    else:
+        return host_or_default
+
+
 def post(
     api_key: str, host: Optional[str] = None, path=None, gzip: bool = False, timeout: int = 15, **kwargs
 ) -> requests.Response:
@@ -50,11 +64,6 @@ def _process_response(
     res: requests.Response, success_message: str, *, return_json: bool = True
 ) -> Union[requests.Response, Any]:
     log = logging.getLogger("posthog")
-    if not res:
-        raise APIError(
-            "N/A",
-            "Error when fetching PostHog API, please make sure you are using your public project token/key and not a private API key.",
-        )
     if res.status_code == 200:
         log.debug(success_message)
         return res.json() if return_json else res
@@ -62,13 +71,13 @@ def _process_response(
         payload = res.json()
         log.debug("received response: %s", payload)
         raise APIError(res.status_code, payload["detail"])
-    except ValueError:
+    except (KeyError, ValueError):
         raise APIError(res.status_code, res.text)
 
 
 def decide(api_key: str, host: Optional[str] = None, gzip: bool = False, timeout: int = 15, **kwargs) -> Any:
     """Post the `kwargs to the decide API endpoint"""
-    res = post(api_key, host, "/decide/?v=2", gzip, timeout, **kwargs)
+    res = post(api_key, host, "/decide/?v=3", gzip, timeout, **kwargs)
     return _process_response(res, success_message="Feature flags decided successfully")
 
 

posthog/sentry/django.py

@@ -11,7 +11,7 @@ def get_distinct_id(request):
         return None
     try:
         return GET_DISTINCT_ID(request)
-    except:
+    except:  # noqa: E722
         return None
 
 

posthog/sentry/posthog_integration.py

@@ -1,5 +1,4 @@
 from sentry_sdk._types import MYPY
-from sentry_sdk.client import Client
 from sentry_sdk.hub import Hub
 from sentry_sdk.integrations import Integration
 from sentry_sdk.scope import add_global_event_processor
@@ -10,9 +9,9 @@ from posthog.request import DEFAULT_HOST
 from posthog.sentry import POSTHOG_ID_TAG
 
 if MYPY:
-    from typing import Any, Dict, Optional
+    from typing import Optional  # noqa: F401
 
-    from sentry_sdk._types import Event, Hint
+    from sentry_sdk._types import Event, Hint  # noqa: F401
 
 
 class PostHogIntegration(Integration):
@@ -44,9 +43,9 @@ class PostHogIntegration(Integration):
                             not not Hub.current.client.dsn and Dsn(Hub.current.client.dsn).project_id
                         )
                         if project_id:
-                            properties[
-                                "$sentry_url"
-                            ] = f"{PostHogIntegration.prefix}{PostHogIntegration.organization}/issues/?project={project_id}&query={event['event_id']}"
+                            properties["$sentry_url"] = (
+                                f"{PostHogIntegration.prefix}{PostHogIntegration.organization}/issues/?project={project_id}&query={event['event_id']}"
+                            )
 
                     posthog.capture(posthog_distinct_id, "$exception", properties)
 

posthog/test/test_client.py

@@ -1,28 +1,38 @@
 import time
 import unittest
-from datetime import date, datetime
-from unittest.mock import MagicMock
+from datetime import datetime
 from uuid import uuid4
 
 import mock
 import six
-from freezegun import freeze_time
 
 from posthog.client import Client
-from posthog.request import APIError
-from posthog.test.test_utils import TEST_API_KEY
+from posthog.test.test_utils import FAKE_TEST_API_KEY
 from posthog.version import VERSION
 
 
 class TestClient(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        # This ensures no real HTTP POST requests are made
+        cls.client_post_patcher = mock.patch("posthog.client.batch_post")
+        cls.consumer_post_patcher = mock.patch("posthog.consumer.batch_post")
+        cls.client_post_patcher.start()
+        cls.consumer_post_patcher.start()
+
+    @classmethod
+    def tearDownClass(cls):
+        cls.client_post_patcher.stop()
+        cls.consumer_post_patcher.stop()
+
     def set_fail(self, e, batch):
         """Mark the failure handler"""
-        print("FAIL", e, batch)
+        print("FAIL", e, batch)  # noqa: T201
         self.failed = True
 
     def setUp(self):
         self.failed = False
-        self.client = Client(TEST_API_KEY, on_error=self.set_fail)
+        self.client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail)
 
     def test_requires_api_key(self):
         self.assertRaises(AssertionError, Client)
@@ -60,8 +70,7 @@ class TestClient(unittest.TestCase):
         self.assertEqual(msg["properties"]["$lib_version"], VERSION)
 
     def test_basic_capture_with_project_api_key(self):
-
-        client = Client(project_api_key=TEST_API_KEY, on_error=self.set_fail)
+        client = Client(project_api_key=FAKE_TEST_API_KEY, on_error=self.set_fail)
 
         success, msg = client.capture("distinct_id", "python test event")
         client.flush()
@@ -79,7 +88,7 @@ class TestClient(unittest.TestCase):
     def test_basic_capture_with_feature_flags(self, patch_decide):
         patch_decide.return_value = {"featureFlags": {"beta-feature": "random-variant"}}
 
-        client = Client(TEST_API_KEY, on_error=self.set_fail, personal_api_key=TEST_API_KEY)
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, personal_api_key=FAKE_TEST_API_KEY)
         success, msg = client.capture("distinct_id", "python test event", send_feature_flags=True)
         client.flush()
         self.assertTrue(success)
@@ -96,11 +105,271 @@ class TestClient(unittest.TestCase):
 
         self.assertEqual(patch_decide.call_count, 1)
 
+    @mock.patch("posthog.client.decide")
+    def test_basic_capture_with_locally_evaluated_feature_flags(self, patch_decide):
+        patch_decide.return_value = {"featureFlags": {"beta-feature": "random-variant"}}
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, personal_api_key=FAKE_TEST_API_KEY)
+
+        multivariate_flag = {
+            "id": 1,
+            "name": "Beta Feature",
+            "key": "beta-feature-local",
+            "is_simple_flag": False,
+            "active": True,
+            "rollout_percentage": 100,
+            "filters": {
+                "groups": [
+                    {
+                        "properties": [
+                            {"key": "email", "type": "person", "value": "test@posthog.com", "operator": "exact"}
+                        ],
+                        "rollout_percentage": 100,
+                    },
+                    {
+                        "rollout_percentage": 50,
+                    },
+                ],
+                "multivariate": {
+                    "variants": [
+                        {"key": "first-variant", "name": "First Variant", "rollout_percentage": 50},
+                        {"key": "second-variant", "name": "Second Variant", "rollout_percentage": 25},
+                        {"key": "third-variant", "name": "Third Variant", "rollout_percentage": 25},
+                    ]
+                },
+                "payloads": {"first-variant": "some-payload", "third-variant": {"a": "json"}},
+            },
+        }
+        basic_flag = {
+            "id": 1,
+            "name": "Beta Feature",
+            "key": "person-flag",
+            "is_simple_flag": True,
+            "active": True,
+            "filters": {
+                "groups": [
+                    {
+                        "properties": [
+                            {
+                                "key": "region",
+                                "operator": "exact",
+                                "value": ["USA"],
+                                "type": "person",
+                            }
+                        ],
+                        "rollout_percentage": 100,
+                    }
+                ],
+                "payloads": {"true": 300},
+            },
+        }
+        false_flag = {
+            "id": 1,
+            "name": "Beta Feature",
+            "key": "false-flag",
+            "is_simple_flag": True,
+            "active": True,
+            "filters": {
+                "groups": [
+                    {
+                        "properties": [],
+                        "rollout_percentage": 0,
+                    }
+                ],
+                "payloads": {"true": 300},
+            },
+        }
+        client.feature_flags = [multivariate_flag, basic_flag, false_flag]
+
+        success, msg = client.capture("distinct_id", "python test event")
+        client.flush()
+        self.assertTrue(success)
+        self.assertFalse(self.failed)
+
+        self.assertEqual(msg["event"], "python test event")
+        self.assertTrue(isinstance(msg["timestamp"], str))
+        self.assertIsNone(msg.get("uuid"))
+        self.assertEqual(msg["distinct_id"], "distinct_id")
+        self.assertEqual(msg["properties"]["$lib"], "posthog-python")
+        self.assertEqual(msg["properties"]["$lib_version"], VERSION)
+        self.assertEqual(msg["properties"]["$feature/beta-feature-local"], "third-variant")
+        self.assertEqual(msg["properties"]["$feature/false-flag"], False)
+        self.assertEqual(msg["properties"]["$active_feature_flags"], ["beta-feature-local"])
+        assert "$feature/beta-feature" not in msg["properties"]
+
+        self.assertEqual(patch_decide.call_count, 0)
+
+        # test that flags are not evaluated without local evaluation
+        client.feature_flags = []
+        success, msg = client.capture("distinct_id", "python test event")
+        client.flush()
+        self.assertTrue(success)
+        self.assertFalse(self.failed)
+        assert "$feature/beta-feature" not in msg["properties"]
+        assert "$feature/beta-feature-local" not in msg["properties"]
+        assert "$feature/false-flag" not in msg["properties"]
+        assert "$active_feature_flags" not in msg["properties"]
+
+    @mock.patch("posthog.client.decide")
+    def test_dont_override_capture_with_local_flags(self, patch_decide):
+        patch_decide.return_value = {"featureFlags": {"beta-feature": "random-variant"}}
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, personal_api_key=FAKE_TEST_API_KEY)
+
+        multivariate_flag = {
+            "id": 1,
+            "name": "Beta Feature",
+            "key": "beta-feature-local",
+            "is_simple_flag": False,
+            "active": True,
+            "rollout_percentage": 100,
+            "filters": {
+                "groups": [
+                    {
+                        "properties": [
+                            {"key": "email", "type": "person", "value": "test@posthog.com", "operator": "exact"}
+                        ],
+                        "rollout_percentage": 100,
+                    },
+                    {
+                        "rollout_percentage": 50,
+                    },
+                ],
+                "multivariate": {
+                    "variants": [
+                        {"key": "first-variant", "name": "First Variant", "rollout_percentage": 50},
+                        {"key": "second-variant", "name": "Second Variant", "rollout_percentage": 25},
+                        {"key": "third-variant", "name": "Third Variant", "rollout_percentage": 25},
+                    ]
+                },
+                "payloads": {"first-variant": "some-payload", "third-variant": {"a": "json"}},
+            },
+        }
+        basic_flag = {
+            "id": 1,
+            "name": "Beta Feature",
+            "key": "person-flag",
+            "is_simple_flag": True,
+            "active": True,
+            "filters": {
+                "groups": [
+                    {
+                        "properties": [
+                            {
+                                "key": "region",
+                                "operator": "exact",
+                                "value": ["USA"],
+                                "type": "person",
+                            }
+                        ],
+                        "rollout_percentage": 100,
+                    }
+                ],
+                "payloads": {"true": 300},
+            },
+        }
+        client.feature_flags = [multivariate_flag, basic_flag]
+
+        success, msg = client.capture(
+            "distinct_id", "python test event", {"$feature/beta-feature-local": "my-custom-variant"}
+        )
+        client.flush()
+        self.assertTrue(success)
+        self.assertFalse(self.failed)
+
+        self.assertEqual(msg["event"], "python test event")
+        self.assertTrue(isinstance(msg["timestamp"], str))
+        self.assertIsNone(msg.get("uuid"))
+        self.assertEqual(msg["distinct_id"], "distinct_id")
+        self.assertEqual(msg["properties"]["$lib"], "posthog-python")
+        self.assertEqual(msg["properties"]["$lib_version"], VERSION)
+        self.assertEqual(msg["properties"]["$feature/beta-feature-local"], "my-custom-variant")
+        self.assertEqual(msg["properties"]["$active_feature_flags"], ["beta-feature-local"])
+        assert "$feature/beta-feature" not in msg["properties"]
+        assert "$feature/person-flag" not in msg["properties"]
+
+        self.assertEqual(patch_decide.call_count, 0)
+
+    @mock.patch("posthog.client.decide")
+    def test_basic_capture_with_feature_flags_returns_active_only(self, patch_decide):
+        patch_decide.return_value = {
+            "featureFlags": {"beta-feature": "random-variant", "alpha-feature": True, "off-feature": False}
+        }
+
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, personal_api_key=FAKE_TEST_API_KEY)
+        success, msg = client.capture("distinct_id", "python test event", send_feature_flags=True)
+        client.flush()
+        self.assertTrue(success)
+        self.assertFalse(self.failed)
+
+        self.assertEqual(msg["event"], "python test event")
+        self.assertTrue(isinstance(msg["timestamp"], str))
+        self.assertIsNone(msg.get("uuid"))
+        self.assertEqual(msg["distinct_id"], "distinct_id")
+        self.assertTrue(msg["properties"]["$geoip_disable"])
+        self.assertEqual(msg["properties"]["$lib"], "posthog-python")
+        self.assertEqual(msg["properties"]["$lib_version"], VERSION)
+        self.assertEqual(msg["properties"]["$feature/beta-feature"], "random-variant")
+        self.assertEqual(msg["properties"]["$feature/alpha-feature"], True)
+        self.assertEqual(msg["properties"]["$active_feature_flags"], ["beta-feature", "alpha-feature"])
+
+        self.assertEqual(patch_decide.call_count, 1)
+        patch_decide.assert_called_with(
+            "random_key",
+            "https://us-api.i.posthog.com",
+            timeout=3,
+            distinct_id="distinct_id",
+            groups={},
+            person_properties=None,
+            group_properties=None,
+            disable_geoip=True,
+        )
+
+    @mock.patch("posthog.client.decide")
+    def test_basic_capture_with_feature_flags_and_disable_geoip_returns_correctly(self, patch_decide):
+        patch_decide.return_value = {
+            "featureFlags": {"beta-feature": "random-variant", "alpha-feature": True, "off-feature": False}
+        }
+
+        client = Client(
+            FAKE_TEST_API_KEY,
+            host="https://app.posthog.com",
+            on_error=self.set_fail,
+            personal_api_key=FAKE_TEST_API_KEY,
+            disable_geoip=True,
+            feature_flags_request_timeout_seconds=12,
+        )
+        success, msg = client.capture("distinct_id", "python test event", send_feature_flags=True, disable_geoip=False)
+        client.flush()
+        self.assertTrue(success)
+        self.assertFalse(self.failed)
+
+        self.assertEqual(msg["event"], "python test event")
+        self.assertTrue(isinstance(msg["timestamp"], str))
+        self.assertIsNone(msg.get("uuid"))
+        self.assertTrue("$geoip_disable" not in msg["properties"])
+        self.assertEqual(msg["distinct_id"], "distinct_id")
+        self.assertEqual(msg["properties"]["$lib"], "posthog-python")
+        self.assertEqual(msg["properties"]["$lib_version"], VERSION)
+        self.assertEqual(msg["properties"]["$feature/beta-feature"], "random-variant")
+        self.assertEqual(msg["properties"]["$feature/alpha-feature"], True)
+        self.assertEqual(msg["properties"]["$active_feature_flags"], ["beta-feature", "alpha-feature"])
+
+        self.assertEqual(patch_decide.call_count, 1)
+        patch_decide.assert_called_with(
+            "random_key",
+            "https://us-api.i.posthog.com",
+            timeout=12,
+            distinct_id="distinct_id",
+            groups={},
+            person_properties=None,
+            group_properties=None,
+            disable_geoip=False,
+        )
+
     @mock.patch("posthog.client.decide")
     def test_basic_capture_with_feature_flags_switched_off_doesnt_send_them(self, patch_decide):
         patch_decide.return_value = {"featureFlags": {"beta-feature": "random-variant"}}
 
-        client = Client(TEST_API_KEY, on_error=self.set_fail, personal_api_key=TEST_API_KEY)
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, personal_api_key=FAKE_TEST_API_KEY)
         success, msg = client.capture("distinct_id", "python test event", send_feature_flags=False)
         client.flush()
         self.assertTrue(success)
@@ -117,29 +386,6 @@ class TestClient(unittest.TestCase):
 
         self.assertEqual(patch_decide.call_count, 0)
 
-    @mock.patch("posthog.client.decide")
-    def test_basic_capture_with_feature_flags_without_api_key(self, patch_decide):
-        patch_decide.return_value = {"featureFlags": {"beta-feature": "random-variant"}}
-
-        client = Client(project_api_key=TEST_API_KEY, on_error=self.set_fail)
-        client.log = MagicMock()
-        success, msg = client.capture("distinct_id", "python test event", send_feature_flags=True)
-        client.flush()
-        self.assertTrue(success)
-        self.assertFalse(self.failed)
-
-        self.assertEqual(msg["event"], "python test event")
-        self.assertTrue(isinstance(msg["timestamp"], str))
-        self.assertIsNone(msg.get("uuid"))
-        self.assertEqual(msg["distinct_id"], "distinct_id")
-        self.assertEqual(msg["properties"]["$lib"], "posthog-python")
-        self.assertEqual(msg["properties"]["$lib_version"], VERSION)
-
-        self.assertEqual(client.log.exception.call_count, 1)
-        client.log.exception.assert_called_with(
-            "[FEATURE FLAGS] Unable to get feature variants: You have to specify a personal_api_key to use feature flags."
-        )
-
     def test_stringifies_distinct_id(self):
         # A large number that loses precision in node:
         # node -e "console.log(157963456373623802 + 1)" > 157963456373623800
@@ -285,6 +531,7 @@ class TestClient(unittest.TestCase):
                 "$group_set": {},
                 "$lib": "posthog-python",
                 "$lib_version": VERSION,
+                "$geoip_disable": True,
             },
         )
         self.assertTrue(isinstance(msg["timestamp"], str))
@@ -306,6 +553,7 @@ class TestClient(unittest.TestCase):
                 "$group_set": {"trait": "value"},
                 "$lib": "posthog-python",
                 "$lib_version": VERSION,
+                "$geoip_disable": True,
             },
         )
         self.assertEqual(msg["timestamp"], "2014-09-03T00:00:00+00:00")
@@ -387,7 +635,7 @@ class TestClient(unittest.TestCase):
             self.assertFalse(consumer.is_alive())
 
     def test_synchronous(self):
-        client = Client(TEST_API_KEY, sync_mode=True)
+        client = Client(FAKE_TEST_API_KEY, sync_mode=True)
 
         success, message = client.identify("distinct_id")
         self.assertFalse(client.consumers)
@@ -395,7 +643,7 @@ class TestClient(unittest.TestCase):
         self.assertTrue(success)
 
     def test_overflow(self):
-        client = Client(TEST_API_KEY, max_queue_size=1)
+        client = Client(FAKE_TEST_API_KEY, max_queue_size=1)
         # Ensure consumer thread is no longer uploading
         client.join()
 
@@ -418,14 +666,14 @@ class TestClient(unittest.TestCase):
         Client("bad_key", debug=True)
 
     def test_gzip(self):
-        client = Client(TEST_API_KEY, on_error=self.fail, gzip=True)
+        client = Client(FAKE_TEST_API_KEY, on_error=self.fail, gzip=True)
         for _ in range(10):
             client.identify("distinct_id", {"trait": "value"})
         client.flush()
         self.assertFalse(self.failed)
 
     def test_user_defined_flush_at(self):
-        client = Client(TEST_API_KEY, on_error=self.fail, flush_at=10, flush_interval=3)
+        client = Client(FAKE_TEST_API_KEY, on_error=self.fail, flush_at=10, flush_interval=3)
 
         def mock_post_fn(*args, **kwargs):
             self.assertEqual(len(kwargs["batch"]), 10)
@@ -439,138 +687,135 @@ class TestClient(unittest.TestCase):
             self.assertEqual(mock_post.call_count, 2)
 
     def test_user_defined_timeout(self):
-        client = Client(TEST_API_KEY, timeout=10)
+        client = Client(FAKE_TEST_API_KEY, timeout=10)
         for consumer in client.consumers:
             self.assertEqual(consumer.timeout, 10)
 
     def test_default_timeout_15(self):
-        client = Client(TEST_API_KEY)
+        client = Client(FAKE_TEST_API_KEY)
         for consumer in client.consumers:
             self.assertEqual(consumer.timeout, 15)
 
-    @mock.patch("posthog.client.Poller")
-    @mock.patch("posthog.client.get")
-    def test_load_feature_flags(self, patch_get, patch_poll):
-        patch_get.return_value = {
-            "results": [
-                {"id": 1, "name": "Beta Feature", "key": "beta-feature", "active": True},
-                {"id": 2, "name": "Alpha Feature", "key": "alpha-feature", "active": False},
-            ]
+    def test_disabled(self):
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, disabled=True)
+        success, msg = client.capture("distinct_id", "python test event")
+        client.flush()
+        self.assertFalse(success)
+        self.assertFalse(self.failed)
+
+        self.assertEqual(msg, "disabled")
+
+    @mock.patch("posthog.client.decide")
+    def test_disabled_with_feature_flags(self, patch_decide):
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, disabled=True)
+
+        response = client.get_feature_flag("beta-feature", "12345")
+        self.assertIsNone(response)
+        patch_decide.assert_not_called()
+
+        response = client.feature_enabled("beta-feature", "12345")
+        self.assertIsNone(response)
+        patch_decide.assert_not_called()
+
+        response = client.get_all_flags("12345")
+        self.assertIsNone(response)
+        patch_decide.assert_not_called()
+
+        response = client.get_feature_flag_payload("key", "12345")
+        self.assertIsNone(response)
+        patch_decide.assert_not_called()
+
+        response = client.get_all_flags_and_payloads("12345")
+        self.assertEqual(response, {"featureFlags": None, "featureFlagPayloads": None})
+        patch_decide.assert_not_called()
+
+        # no capture calls
+        self.assertTrue(client.queue.empty())
+
+    def test_enabled_to_disabled(self):
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, disabled=False)
+        success, msg = client.capture("distinct_id", "python test event")
+        client.flush()
+
+        self.assertTrue(success)
+        self.assertFalse(self.failed)
+        self.assertEqual(msg["event"], "python test event")
+
+        client.disabled = True
+        success, msg = client.capture("distinct_id", "python test event")
+        client.flush()
+        self.assertFalse(success)
+        self.assertFalse(self.failed)
+
+        self.assertEqual(msg, "disabled")
+
+    def test_disable_geoip_default_on_events(self):
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, disable_geoip=True)
+        _, capture_msg = client.capture("distinct_id", "python test event")
+        client.flush()
+        self.assertEqual(capture_msg["properties"]["$geoip_disable"], True)
+
+        _, identify_msg = client.identify("distinct_id", {"trait": "value"})
+        client.flush()
+        self.assertEqual(identify_msg["properties"]["$geoip_disable"], True)
+
+    def test_disable_geoip_override_on_events(self):
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, disable_geoip=False)
+        _, capture_msg = client.set("distinct_id", {"a": "b", "c": "d"}, disable_geoip=True)
+        client.flush()
+        self.assertEqual(capture_msg["properties"]["$geoip_disable"], True)
+
+        _, identify_msg = client.page("distinct_id", "http://a.com", {"trait": "value"}, disable_geoip=False)
+        client.flush()
+        self.assertEqual("$geoip_disable" not in identify_msg["properties"], True)
+
+    def test_disable_geoip_method_overrides_init_on_events(self):
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, disable_geoip=True)
+        _, msg = client.capture("distinct_id", "python test event", disable_geoip=False)
+        client.flush()
+        self.assertTrue("$geoip_disable" not in msg["properties"])
+
+    @mock.patch("posthog.client.decide")
+    def test_disable_geoip_default_on_decide(self, patch_decide):
+        patch_decide.return_value = {
+            "featureFlags": {"beta-feature": "random-variant", "alpha-feature": True, "off-feature": False}
         }
-        client = Client(TEST_API_KEY, personal_api_key="test")
-        with freeze_time("2020-01-01T12:01:00.0000Z"):
-            client.load_feature_flags()
-        self.assertEqual(len(client.feature_flags), 1)
-        self.assertEqual(client.feature_flags[0]["key"], "beta-feature")
-        self.assertEqual(client._last_feature_flag_poll.isoformat(), "2020-01-01T12:01:00+00:00")
-        self.assertEqual(patch_poll.call_count, 1)
-
-    def test_load_feature_flags_wrong_key(self):
-        client = Client(TEST_API_KEY, personal_api_key=TEST_API_KEY)
-        with freeze_time("2020-01-01T12:01:00.0000Z"):
-            self.assertRaises(APIError, client.load_feature_flags)
-
-    @mock.patch("posthog.client.decide")
-    @mock.patch("posthog.client.get")
-    def test_feature_enabled_simple(self, patch_get, patch_decide):
-        client = Client(TEST_API_KEY)
-        client.feature_flags = [
-            {"id": 1, "name": "Beta Feature", "key": "beta-feature", "is_simple_flag": True, "rollout_percentage": 100}
-        ]
-        self.assertTrue(client.feature_enabled("beta-feature", "distinct_id"))
-        self.assertEqual(patch_decide.call_count, 0)
-
-    @mock.patch("posthog.client.decide")
-    @mock.patch("posthog.client.get")
-    def test_feature_enabled_simple_is_false(self, patch_get, patch_decide):
-        client = Client(TEST_API_KEY)
-        client.feature_flags = [
-            {"id": 1, "name": "Beta Feature", "key": "beta-feature", "is_simple_flag": True, "rollout_percentage": 0}
-        ]
-        self.assertFalse(client.feature_enabled("beta-feature", "distinct_id"))
-        self.assertEqual(patch_decide.call_count, 0)
-
-    @mock.patch("posthog.client.decide")
-    @mock.patch("posthog.client.get")
-    def test_feature_enabled_simple_is_true_when_rollout_is_undefined(self, patch_get, patch_decide):
-        client = Client(TEST_API_KEY)
-        client.feature_flags = [
-            {"id": 1, "name": "Beta Feature", "key": "beta-feature", "is_simple_flag": True, "rollout_percentage": None}
-        ]
-        self.assertTrue(client.feature_enabled("beta-feature", "distinct_id"))
-        self.assertEqual(patch_decide.call_count, 0)
-
-    @mock.patch("posthog.client.get")
-    def test_feature_enabled_simple_with_project_api_key(self, patch_get):
-        client = Client(project_api_key=TEST_API_KEY, on_error=self.set_fail)
-        client.feature_flags = [
-            {"id": 1, "name": "Beta Feature", "key": "beta-feature", "is_simple_flag": True, "rollout_percentage": 100}
-        ]
-        self.assertTrue(client.feature_enabled("beta-feature", "distinct_id"))
-
-    @mock.patch("posthog.client.decide")
-    def test_feature_enabled_request(self, patch_decide):
-        patch_decide.return_value = {"featureFlags": {"beta-feature": True}}
-        client = Client(TEST_API_KEY, personal_api_key="test")
-        client.feature_flags = [
-            {"id": 1, "name": "Beta Feature", "key": "beta-feature", "is_simple_flag": False, "rollout_percentage": 100}
-        ]
-        self.assertTrue(client.feature_enabled("beta-feature", "distinct_id"))
-
-    @mock.patch("posthog.client.decide")
-    def test_feature_enabled_request_multi_variate(self, patch_decide):
-        patch_decide.return_value = {"featureFlags": {"beta-feature": "variant-1"}}
-        client = Client(TEST_API_KEY, personal_api_key="test")
-        client.feature_flags = [
-            {"id": 1, "name": "Beta Feature", "key": "beta-feature", "is_simple_flag": False, "rollout_percentage": 100}
-        ]
-        self.assertEqual(client.feature_enabled("beta-feature", "distinct_id"), "variant-1")
-
-    @mock.patch("posthog.client.get")
-    def test_feature_enabled_simple_without_rollout_percentage(self, patch_get):
-        client = Client(TEST_API_KEY)
-        client.feature_flags = [{"id": 1, "name": "Beta Feature", "key": "beta-feature", "is_simple_flag": True}]
-        self.assertTrue(client.feature_enabled("beta-feature", "distinct_id"))
-
-    @mock.patch("posthog.client.get")
-    def test_feature_enabled_simple_with_none_rollout_percentage(self, patch_get):
-        client = Client(TEST_API_KEY)
-        client.feature_flags = [
-            {"id": 1, "name": "Beta Feature", "key": "beta-feature", "is_simple_flag": True, "rollout_percantage": None}
-        ]
-        self.assertTrue(client.feature_enabled("beta-feature", "distinct_id"))
-
-    @mock.patch("posthog.client.Poller")
-    @mock.patch("posthog.client.decide")
-    def test_feature_enabled_doesnt_exist(self, patch_decide, patch_poll):
-        patch_decide.return_value = {"featureFlags": {}}
-        client = Client(TEST_API_KEY)
-        client.feature_flags = []
-
-        self.assertFalse(client.feature_enabled("doesnt-exist", "distinct_id"))
-        self.assertTrue(client.feature_enabled("doesnt-exist", "distinct_id", True))
-
-    @mock.patch("posthog.client.Poller")
-    @mock.patch("posthog.client.decide")
-    def test_personal_api_key_doesnt_exist(self, patch_decide, patch_poll):
-        client = Client(TEST_API_KEY, personal_api_key="test")
-        client.feature_flags = []
-
-        patch_decide.return_value = {"featureFlags": {"feature-flag": True}}
-
-        self.assertTrue(client.feature_enabled("feature-flag", "distinct_id"))
-
-    @mock.patch("posthog.client.Poller")
-    @mock.patch("posthog.client.get")
-    def test_load_feature_flags_error(self, patch_get, patch_poll):
-        def raise_effect():
-            raise Exception("http exception")
-
-        patch_get.return_value.raiseError.side_effect = raise_effect
-        client = Client(TEST_API_KEY, personal_api_key="test")
-        client.feature_flags = []
-
-        self.assertFalse(client.feature_enabled("doesnt-exist", "distinct_id"))
+        client = Client(FAKE_TEST_API_KEY, on_error=self.set_fail, disable_geoip=False)
+        client.get_feature_flag("random_key", "some_id", disable_geoip=True)
+        patch_decide.assert_called_with(
+            "random_key",
+            "https://us-api.i.posthog.com",
+            timeout=3,
+            distinct_id="some_id",
+            groups={},
+            person_properties={"distinct_id": "some_id"},
+            group_properties={},
+            disable_geoip=True,
+        )
+        patch_decide.reset_mock()
+        client.feature_enabled("random_key", "feature_enabled_distinct_id", disable_geoip=True)
+        patch_decide.assert_called_with(
+            "random_key",
+            "https://us-api.i.posthog.com",
+            timeout=3,
+            distinct_id="feature_enabled_distinct_id",
+            groups={},
+            person_properties={"distinct_id": "feature_enabled_distinct_id"},
+            group_properties={},
+            disable_geoip=True,
+        )
+        patch_decide.reset_mock()
+        client.get_all_flags_and_payloads("all_flags_payloads_id")
+        patch_decide.assert_called_with(
+            "random_key",
+            "https://us-api.i.posthog.com",
+            timeout=3,
+            distinct_id="all_flags_payloads_id",
+            groups={},
+            person_properties={"distinct_id": "all_flags_payloads_id"},
+            group_properties={},
+            disable_geoip=False,
+        )
 
     @mock.patch("posthog.client.Poller")
     @mock.patch("posthog.client.get")
@@ -579,7 +824,74 @@ class TestClient(unittest.TestCase):
             raise Exception("http exception")
 
         patch_get.return_value.raiseError.side_effect = raise_effect
-        client = Client(TEST_API_KEY, personal_api_key="test")
+        client = Client(FAKE_TEST_API_KEY, personal_api_key="test")
         client.feature_flags = [{"key": "example", "is_simple_flag": False}]
 
         self.assertFalse(client.feature_enabled("example", "distinct_id"))
+
+    @mock.patch("posthog.client.decide")
+    def test_default_properties_get_added_properly(self, patch_decide):
+        patch_decide.return_value = {
+            "featureFlags": {"beta-feature": "random-variant", "alpha-feature": True, "off-feature": False}
+        }
+        client = Client(FAKE_TEST_API_KEY, host="http://app2.posthog.com", on_error=self.set_fail, disable_geoip=False)
+        client.get_feature_flag(
+            "random_key",
+            "some_id",
+            groups={"company": "id:5", "instance": "app.posthog.com"},
+            person_properties={"x1": "y1"},
+            group_properties={"company": {"x": "y"}},
+        )
+        patch_decide.assert_called_with(
+            "random_key",
+            "http://app2.posthog.com",
+            timeout=3,
+            distinct_id="some_id",
+            groups={"company": "id:5", "instance": "app.posthog.com"},
+            person_properties={"distinct_id": "some_id", "x1": "y1"},
+            group_properties={
+                "company": {"$group_key": "id:5", "x": "y"},
+                "instance": {"$group_key": "app.posthog.com"},
+            },
+            disable_geoip=False,
+        )
+
+        patch_decide.reset_mock()
+        client.get_feature_flag(
+            "random_key",
+            "some_id",
+            groups={"company": "id:5", "instance": "app.posthog.com"},
+            person_properties={"distinct_id": "override"},
+            group_properties={
+                "company": {
+                    "$group_key": "group_override",
+                }
+            },
+        )
+        patch_decide.assert_called_with(
+            "random_key",
+            "http://app2.posthog.com",
+            timeout=3,
+            distinct_id="some_id",
+            groups={"company": "id:5", "instance": "app.posthog.com"},
+            person_properties={"distinct_id": "override"},
+            group_properties={
+                "company": {"$group_key": "group_override"},
+                "instance": {"$group_key": "app.posthog.com"},
+            },
+            disable_geoip=False,
+        )
+
+        patch_decide.reset_mock()
+        # test nones
+        client.get_all_flags_and_payloads("some_id", groups={}, person_properties=None, group_properties=None)
+        patch_decide.assert_called_with(
+            "random_key",
+            "http://app2.posthog.com",
+            timeout=3,
+            distinct_id="some_id",
+            groups={},
+            person_properties={"distinct_id": "some_id"},
+            group_properties={},
+            disable_geoip=False,
+        )

posthog/test/test_consumer.py

@@ -145,15 +145,20 @@ class TestConsumer(unittest.TestCase):
     def test_max_batch_size(self):
         q = Queue()
         consumer = Consumer(q, TEST_API_KEY, flush_at=100000, flush_interval=3)
-        track = {"type": "track", "event": "python event", "distinct_id": "distinct_id"}
+        properties = {}
+        for n in range(0, 500):
+            properties[str(n)] = "one_long_property_value_to_build_a_big_event"
+        track = {"type": "track", "event": "python event", "distinct_id": "distinct_id", "properties": properties}
         msg_size = len(json.dumps(track).encode())
-        # number of messages in a maximum-size batch
-        n_msgs = int(475000 / msg_size)
+        # Let's capture 8MB of data to trigger two batches
+        n_msgs = int(8_000_000 / msg_size)
 
         def mock_post_fn(_, data, **kwargs):
             res = mock.Mock()
             res.status_code = 200
-            self.assertTrue(len(data.encode()) < 500000, "batch size (%d) exceeds 500KB limit" % len(data.encode()))
+            request_size = len(data.encode())
+            # Batches close after the first message bringing it bigger than BATCH_SIZE_LIMIT, let's add 10% of margin
+            self.assertTrue(request_size < (5 * 1024 * 1024) * 1.1, "batch size (%d) higher than limit" % request_size)
             return res
 
         with mock.patch("posthog.request._session.post", side_effect=mock_post_fn) as mock_post:

posthog/test/test_module.py

@@ -1,40 +1,48 @@
 import unittest
 
-import posthog
+from posthog import Posthog
 
 
 class TestModule(unittest.TestCase):
+    posthog = None
+
+    def _assert_enqueue_result(self, result):
+        self.assertEqual(type(result[0]), bool)
+        self.assertEqual(type(result[1]), dict)
+
     def failed(self):
         self.failed = True
 
     def setUp(self):
         self.failed = False
-        posthog.api_key = "testsecret"
-        posthog.on_error = self.failed
+        self.posthog = Posthog("testsecret", host="http://localhost:8000", on_error=self.failed)
 
     def test_no_api_key(self):
-        posthog.api_key = None
-        self.assertRaises(Exception, posthog.capture)
+        self.posthog.api_key = None
+        self.assertRaises(Exception, self.posthog.capture)
 
     def test_no_host(self):
-        posthog.host = None
-        self.assertRaises(Exception, posthog.capture)
+        self.posthog.host = None
+        self.assertRaises(Exception, self.posthog.capture)
 
     def test_track(self):
-        posthog.capture("distinct_id", "python module event")
-        posthog.flush()
+        res = self.posthog.capture("distinct_id", "python module event")
+        self._assert_enqueue_result(res)
+        self.posthog.flush()
 
     def test_identify(self):
-        posthog.identify("distinct_id", {"email": "user@email.com"})
-        posthog.flush()
+        res = self.posthog.identify("distinct_id", {"email": "user@email.com"})
+        self._assert_enqueue_result(res)
+        self.posthog.flush()
 
     def test_alias(self):
-        posthog.alias("previousId", "distinct_id")
-        posthog.flush()
+        res = self.posthog.alias("previousId", "distinct_id")
+        self._assert_enqueue_result(res)
+        self.posthog.flush()
 
     def test_page(self):
-        posthog.page("distinct_id", "https://posthog.com/contact")
-        posthog.flush()
+        self.posthog.page("distinct_id", "https://posthog.com/contact")
+        self.posthog.flush()
 
     def test_flush(self):
-        posthog.flush()
+        self.posthog.flush()

posthog/test/test_request.py

@@ -2,9 +2,10 @@ import json
 import unittest
 from datetime import date, datetime
 
+import pytest
 import requests
 
-from posthog.request import DatetimeSerializer, batch_post
+from posthog.request import DatetimeSerializer, batch_post, determine_server_host
 from posthog.test.test_utils import TEST_API_KEY
 
 
@@ -42,3 +43,26 @@ class TestRequests(unittest.TestCase):
             batch_post(
                 "key", batch=[{"distinct_id": "distinct_id", "event": "python event", "type": "track"}], timeout=0.0001
             )
+
+
+@pytest.mark.parametrize(
+    "host, expected",
+    [
+        ("https://t.posthog.com", "https://t.posthog.com"),
+        ("https://t.posthog.com/", "https://t.posthog.com/"),
+        ("t.posthog.com", "t.posthog.com"),
+        ("t.posthog.com/", "t.posthog.com/"),
+        ("https://us.posthog.com.rg.proxy.com", "https://us.posthog.com.rg.proxy.com"),
+        ("app.posthog.com", "app.posthog.com"),
+        ("eu.posthog.com", "eu.posthog.com"),
+        ("https://app.posthog.com", "https://us-api.i.posthog.com"),
+        ("https://eu.posthog.com", "https://eu-api.i.posthog.com"),
+        ("https://us.posthog.com", "https://us-api.i.posthog.com"),
+        ("https://app.posthog.com/", "https://us-api.i.posthog.com"),
+        ("https://eu.posthog.com/", "https://eu-api.i.posthog.com"),
+        ("https://us.posthog.com/", "https://us-api.i.posthog.com"),
+        (None, "https://us-api.i.posthog.com"),
+    ],
+)
+def test_routing_to_custom_host(host, expected):
+    assert determine_server_host(host) == expected

posthog/test/test_utils.py

@@ -9,6 +9,7 @@ from dateutil.tz import tzutc
 from posthog import utils
 
 TEST_API_KEY = "kOOlRy2QlMY9jHZQv0bKz0FZyazBUoY8Arj0lFVNjs4"
+FAKE_TEST_API_KEY = "random_key"
 
 
 class TestUtils(unittest.TestCase):
@@ -79,3 +80,22 @@ class TestUtils(unittest.TestCase):
     def test_remove_slash(self):
         self.assertEqual("http://posthog.io", utils.remove_trailing_slash("http://posthog.io/"))
         self.assertEqual("http://posthog.io", utils.remove_trailing_slash("http://posthog.io"))
+
+
+class TestSizeLimitedDict(unittest.TestCase):
+    def test_size_limited_dict(self):
+        size = 10
+        values = utils.SizeLimitedDict(size, lambda _: -1)
+
+        for i in range(100):
+            values[i] = i
+
+            self.assertEqual(values[i], i)
+            self.assertEqual(len(values), i % size + 1)
+
+            if i % size == 0:
+                # old numbers should've been removed
+                self.assertIsNone(values.get(i - 1))
+                self.assertIsNone(values.get(i - 3))
+                self.assertIsNone(values.get(i - 5))
+                self.assertIsNone(values.get(i - 9))

posthog/utils.py

@@ -1,6 +1,8 @@
 import logging
 import numbers
-from datetime import date, datetime
+import re
+from collections import defaultdict
+from datetime import date, datetime, timezone
 from decimal import Decimal
 from uuid import UUID
 
@@ -87,3 +89,29 @@ def _coerce_unicode(cmplx):
         log.warning("Error decoding: %s", item)
         return None
     return item
+
+
+def is_valid_regex(value) -> bool:
+    try:
+        re.compile(value)
+        return True
+    except re.error:
+        return False
+
+
+class SizeLimitedDict(defaultdict):
+    def __init__(self, max_size, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.max_size = max_size
+
+    def __setitem__(self, key, value):
+        if len(self) >= self.max_size:
+            self.clear()
+
+        super().__setitem__(key, value)
+
+
+def convert_to_datetime_aware(date_obj):
+    if date_obj.tzinfo is None:
+        date_obj = date_obj.replace(tzinfo=timezone.utc)
+    return date_obj

posthog/version.py

@@ -1,4 +1,4 @@
-VERSION = "1.4.9"
+VERSION = "3.5.0"
 
 if __name__ == "__main__":
-    print(VERSION, end="")
+    print(VERSION, end="")  # noqa: T201

sentry_django_example/sentry_django_example/urls.py

@@ -13,6 +13,7 @@ Including another URLconf
     1. Import the include() function: from django.urls import include, path
     2. Add a URL to urlpatterns:  path('blog/', include('blog.urls'))
 """
+
 from django.contrib import admin
 from django.urls import path
 

setup.py

@@ -14,15 +14,17 @@ long_description = """
 PostHog is developer-friendly, self-hosted product analytics. posthog-python is the python package.
 """
 
-install_requires = ["requests>=2.7,<3.0", "six>=1.5", "monotonic>=1.5", "backoff>=1.10.0,<2.0.0", "python-dateutil>2.1"]
+install_requires = ["requests>=2.7,<3.0", "six>=1.5", "monotonic>=1.5", "backoff>=1.10.0", "python-dateutil>2.1"]
 
 extras_require = {
     "dev": [
         "black",
         "isort",
+        "flake8",
+        "flake8-print",
         "pre-commit",
     ],
-    "test": ["mock>=2.0.0", "freezegun==0.3.15", "pylint", "flake8", "coverage", "pytest"],
+    "test": ["mock>=2.0.0", "freezegun==0.3.15", "pylint", "flake8", "coverage", "pytest", "pytest-timeout"],
     "sentry": ["sentry-sdk", "django"],
 }
 

setup_analytics.py

@@ -14,7 +14,7 @@ long_description = """
 PostHog is developer-friendly, self-hosted product analytics. posthog-python is the python package.
 """
 
-install_requires = ["requests>=2.7,<3.0", "six>=1.5", "monotonic>=1.5", "backoff==1.6.0", "python-dateutil>2.1"]
+install_requires = ["requests>=2.7,<3.0", "six>=1.5", "monotonic>=1.5", "backoff>=1.10.0", "python-dateutil>2.1"]
 
 tests_require = ["mock>=2.0.0"]