Python keyword only arguments

Python Keyword-Only Arguments

Python 3 got new feature called keyword-only arguments. Python has already a support for variables length positional arguments and variable length keyword arguments.

Jump to,

Current Syntax

By variable length positional arguments we can pass any number of positional arguments which are sucked up by a variable provided by syntax ” *varargs “. Also, by the syntax of variable length keyword arguments “*kwargs” we can pass any number of keyword arguments to function which are available as dictionary in the function.

def foo(a, b, *varargs, **kwargs):
print(a, b, varargs, kwargs)

## Call foo
foo(2, 9, 12, 34, x=3, name="bob")

Which outputs, the following. excluding 2, 9 all other positional arguments are sucked up by “vargars” and keywords are by “kwargs”

2 9 (12, 34) {'x': 3, 'name': 'bob'}

Existing syntax doesn’t allow us to use any kind of keyword arguments beyond variable length keyword arguments “**kwargs“.

You can choose to pass arguments by either position or it’s keyword.

One major limit for this syntax is that, all the regular arguments slots must be filled before the Variable Length keyword arguments slot.

Keyword-Only Arguments

This feature allows us to define a function to accept selected arguments by it’s keyword only. Which literally means you have to pass arguments to the function during function call by using it’s keyword only, otherwise it trows an exception. These keyword only arguments will never be automatically filled in by a positional argument.

How it’s useful

  • It improves the readability
  • API functions can accept the critical parameters as keyword only arguments to avoid ambiguity
  • You can create the function with all of it’s arguments as keyword only arguments
  • Functions with keyword only arguments are useful in selected use case scenarios when function is taking huge number of arguments

There are two kind of keyword only arguments. Which are,

  • Non-Default Keyword-Only Arguments
  • Default Keyword-Only Arguments

Non-Default Keyword-Only Arguments

Non-Default keyword-only arguments are keyword-only arguments without any defaults in function definition. If function is defined with non-default keyword-only arguments. Function should be called by passing corresponding arguments (with keywords). See the Syntax for Non-Default Keyword-Only Arguments

Default Keyword-Only Arguments

The name Default Keyword-Only Arguments is itself self explanatory. As it says, These are keyword-only arguments with defaults in the function definition

Keyword-Only Arguments Syntax

The syntax change is fairly very simple, it allows you to define keyword only arguments following variable length positional args *varargs .

def sortwords(*words,  case_sensitive=True):
. . .

Here the argument case_sensitive is keyword-only. Every positional arg you give goes to words argument. The argument case_sensitive is available as it’s default or should be passed by it’s keyword during function call

sortwords("bob", "jack", "rose",  case_sensitive=False)

Le’ts mix up regular positional, variable length positional and keyword-only arguments.

def add(a, b, *more, print_result=True):
...

According to above function definition, argument slots a and b are filled up first and then left over positional arguments are sucked up by argument more. Argument print_result will never be sucked up by vararg ‘more’. It can only be passed as keyword-only argument.

Let’s see the above function in action,

def add(a, b, *more, print_result=True):
result = a + b + sum(more)
if print_result:
print("sum of {} {} {} is {}".format(a, b, ' '.join(map(str, more)), result))
return result

Call above function

>>> add(1, 3, 9) 
sum of 1 3 9 is 13
13
>>> add(2, 9, 12, 3, print_result=False)
26
>>>

Here, did you notice ? the argument “more” will also suck up any erroneous positional arguments.

>>> add(9, 2, 5, 6, 'not sure why im here')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in add
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Since we are performing addition, expected arguments are numbers but passing any other non compatible type causes error. In this case we might need to enforce arguments by it’s type or if you choose not to take or ignore varargs which brings us the following syntax “*” .

Let’s put all together,

def compute(a, b, c, *more, print_result=False, ignore_execeptions=True, **kwargs):
...

Let’s mix up all kind of arguments to see the kind of arguments, order and where these keyword-only arguments fit in the function definition.

Python Function Definition, Arguments Kind and Order
Python Function Definition, Arguments Kind and Order

Let’s see simple example in action to learn how we can use these arguments in combination with others.

def compute(a, b, c, print_args=True, intype=int,  *more, operation, print_result=False, return_type, ignore_exceptions=True, **kwargs):
if print_args:
print(a, b, c, print_args, intype, more, operation, print_result, return_type, ignore_exceptions, kwargs)
result = None
try:
arr = (a, b, c) + more
if operation == "count":
result = len(arr)
elif operation == "sum":
result = sum(arr)
else:
print("Unknow Operation")
raise NotImplementedError("Unknow Operation. Operation Not Implemented")
except:
import sys
print("Error performing operation: {}: {}".format(operation, sys.exc_info()))
if not ignore_exceptions:
raise
if print_result:
print("Result is: {}".format(result))
return result and return_type(result)

Above example may not be serving any logical purpose but intention was to show you the keyword-only arguments in action along with other argument types.

Keyword-Only Arguments with no varargs

Another syntax change is that, you can omit the name for the varargs argument to take keyword-only arguments and raise exception if any extra positional arguments are passed.

def compare(a, b, *, key=None):
. . .

Non-Default Keyword-Only Arguments Syntax

It is not necessary to provide a default value for keyword-only argument in the function definition

def compare(a, b, *, key):
. . .
>>>  compare(2, 3)   # missing non-default keyword-only arg
Traceback (most recent call last):
File "", line 1, in
TypeError: compare() missing 1 required keyword-only argument: 'key'

It trows TypeError exception if keyword-only argument is not passed in the function call. Since it is not having default value given.

Cross-Origin Resource Sharing (CORS) Explained

Cross Origin Resource Sharing

Cross-Origin Resource Sharing (CORS) is the mechanism to allow content loaded from one (main) origin to access the selected resources available from servers at different origin. This mechanism of accessing resources at different origin is called CORS (Cross-Origin Resource Sharing). This is possible by making use of additional HTTP headers by both browser and Cross Origin server.

CORS is the mechanism among others to relax the browsers Same-Origin Policy

Let’s see what is Same-Origin Policy, and how CORS is used to relax this restriction

If you are excited to read through whole article 👍, else jump on to ,

What is Origin ?

Origin
HTTP Origin Explained

An Origin can be defined as combination of Protocol, domain and port . We can also use URI Scheme instead of protocol, like wise hostname instead of domain. The origin describes the origin/location from which the document is loaded. You can access the origin of document in JavaScript as follows

document.location.origin

If port is not mentioned it defaults to 80 . Origin of a document won’t be changed once browser loads a main page/document from the server. On the other hand Origin is also the HTTP Request Header . Which is used to describe the origin of XHR or fetch requests, which is obviously the origin of the document from which script is making requests. Origin header is sent with CORS requests including POST requests. As we have seen above, origin is the combination of Protocol, domain/host and port. Origin doesn’t include any path, port is optional(omitted) if it is 80 .

The Origin header describes from which origin the HTTP requests are coming from to the Cross(different) Origin Server. The Cross Origin server acts based on the kind of Origin header it receives.

What is Same-Origin Policy? Why it is Important ?

Let’s see little bit about Same-Origin Policy before we jump on to CORS which is there to relax it.

The Same-Origin Policy is a security mechanism under which the browser imposes the restriction on scripts running on a document loaded from one origin from accessing the data from another origin. This policy is critical in web security model which prevents malicious scripts running on one document from obtaining access to data on another document of different origin through DOM. It is important, since HTTP is a stateless and connectionless protocol, cookies are used to keep track of the state.

This restriction is only applied to the scripts running on a document from making requests to different origin. But not to the embedded resources like images, CSS and scripts.

Let’s see simple example, what if there is not same-origin policy

Let’s assume you visited a banking website and still logged in. And also assume you went to the website which has some malicious JavaScript code, this could be in a different tab or same tab or different window but same browser. Because you are still logged in on the banking website, the malicious script running on different origin can impersonate you by sending requests to banking site which is at different origin. Since it can do any thing as you could do on the banking site. It can transfer amount, make payments and even download the transaction history. This all happens because the browser sends and receives cookies to/from the banking site based on its domain.

Well, you visiting the malicious site may think it has no access to the banking site cookies. That might be true, that Javascript has no direct access to the cookies of banking site, yet it can send the requests to the path/document at different origin of banking site. Browser attaches the cookies based on domain. Since script impersonates you as a user, it can do anything even CSRF protection would be not that effective.

Same-Origin vs Cross-Origin

Two URLs are said to be same origin only if all protocol, domain and port match in the. When browser makes a request to the URL it checks the origin to apply same-origin policy

Let’s take the Reference URL https://neo.util.tools/bit-toggler/ to check whether if following URLs are same origin or Cross Origin

URL
Same Origin
Cross Origin
Reason
https://neo.util.tools/word-counter/
Same Origin but different path
https://neo.util.tools/stop-watch/
Same Origin but different path
https://util.tools/stop-watch/
Different Domain
http://neo.util.tools/stop-watch/
Different Protocol http
http://getkt.com/
Different Domain & protocol
http://192.168.1.24:8090
Different protocol,host & port

As we have seen above, if origin is determined as Cross Origin. Browser will take necessary measures and impose a restriction on scripts sending the request to Cross Origin. It is absolutely good and secure, but it is always not desirable and too restrictive when we want to communicate legitimately to the remote Cross Origin Server.

There are different ways to relax this browsers Same-Origin Policy . CORS is one of them among other which are, document.domain, Cross-Document Messaging, old schoold JSONP and Web Sockets. So that, we scripts on origin can access the data on Cross-Origin Server securely by sending any kind of HTTP Requests.

Who Uses CORS? When it is Used ?

Well, now almost every web developer uses CORS even if he is not aware. CORS is used by any application which makes use of the following,

  • Web Fonts
  • Oauth & OpenID services
  • API Service Providers
  • Payment Gateways
  • REST API
  • Service providers for web applications like
    • stripe

CORS plays a vital role if you are creating the application by using the model SOA(Service Oriented Applications) . Since whole application is made up of several micro-services as opposed to monolithic application. In which case, browser would need to communicate with several microservices. Ideally each micro-service would be running on it’s own origin.

CORS Functional Overview

Cross-Origin Resource Sharing (CORS) is the mechanism to allow scripts running at one origin in the browser to access the selected resources from web server at different origin. During CORS browser uses additional HTTP Request Headers to inform remote sever at different origin that, from which origin requests are coming from, and additional HTTP Response headers are used by web server at different origin to tell browser if it is having access to the requested resources.

Failure of CORS potentially due to the Cross-Origin server doesn’t approve the origin result in an error. Specifics of this error won’t be available to the JavaScript code for security reason. but, the code gets the signal that there was an error. For debugging to get details of what went wrong we can take a look at browsers console.

It’s browsers responsibility to support CORS and honor the restriction if servers don’t approve the CORS

There are two types of CORS requests.

Based on kind of HTTP method, Mime Type and headers that are used to make HTTP request browser makes one of the following CORS Request

According to the CORS Specification browser would send an additional preflight request before the actual request for HTTP methods that can cause the side effect on server side data to determine if client(origin) is having permission to access/modify the data at cross-origin. Upon approval from the cross-origin server through additional CORS Response Headers, browser will send the actual request. This preflight request which is OPTIONS method, is sent only for HTTP methods that can cause the side effect on server’s data which are other than GET and POST with certain MIME Types.

Cross Origin Request Abstract Flow
Cross Origin Request Abstract Flow

CORS – Simple Request

Some CORS requests don’t trigger preflight request, so we call it as ‘simple request’.

Simple requests are made by browser to cross origin server only if browser think that, the request won’t cause side-effects on server side data. For simple request browser offers very limited access to the scripts on HTTP Response.

Simple Request Conditions

Browser makes “Simple Request” only if following conditions are met,

Only following methods don’t cause prelight request
  • GET
  • HEAD
  • POST with certain mime types
Other than headers that are automatically set by the browser. According the specs, browser make the simple request only if they are one of CORS safelisted Request Headers. Which are,
  • Accept
  • Accept-Language
  • Content-Language
  • Content-Type value should one of see below
  • DRP
  • Downlink
  • Save-Data
  • Viewport-Width
  • Width
Allowed values of the Request Header Content-Type are,
  • application/x-www-form-urlencoded
  • multipart/form-data
  • text/plain
Now even listener should be registered XMLHttpRequestUpload
No ReadableStream associated with Request

If you are using fetch to make CORS request, you can only access the following Standard Response headers because of restriction according to the spec Safe listed Response Headers

  • Cache-Control
  • Content-Language
  • Content-Length
  • Content-Type
  • Expires
  • Last-Modified
  • Pragma

and anything which is not part of forbidden response header names

Simple Request in Action

Simple request makes use of mainly Origin and Access-Control-Allow-Origin request and response headers respectively for CORS to handle privileges.

Let’s see simple exchange of headers for simple request

CORS Simple Request
CORS Simple Request

As you can see in the above picture. Sending request by xhr, fetch or using webfonts in style triggers browser CORS mechanism. Here assume that we are on page of https://getkt.com, from this page we requested font Open Sans from host fonts.googleapis.com. Since we are sending GET request and not used any custom headers. Browser won’t send any preflight request. Browser included “https://getkt.com” into the Origin header. Usually server would include what comes in Origin into Response Header “Access-Control-Allow-Origin” . But here “*” is included in the header, which means all origins are allowed to access the resource.

Let’s try the same thing in your favorite browsers console.

Open console for any website. ideally that website will be the origin of request. Assume we are on https://getkt.com

Origin: https://getkt.com
Resource URL: https://fonts.googleapis.com/css?family=Open+Sans

Here Cross Origin will be https://fonts.googleapis.com with respect to https://getkt.com

Let’s run the following snippet and inspect your self the headers for the success scenario of simple CORS request.

 corsURL = 'https://fonts.googleapis.com/css?family=Open+Sans';
xhr = new XMLHttpRequest();
xhr.open('GET', corsURL, true);
xhr.onload = r => console.log(r);
xhr.send();
CORS simple request browser console inspect

Here is how full exchange of headers between client and server

 GET /css?family=Open+Sans HTTP/1.1
Host: fonts.googleapis.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0
Accept: /
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://getkt.com/
Origin: https://getkt.com
Connection: keep-alive


HTTP/2.0 200 OK
content-type: text/css; charset=utf-8
access-control-allow-origin: *
timing-allow-origin: *
link: https://fonts.gstatic.com; rel=preconnect; crossorigin
strict-transport-security: max-age=31536000
expires: Mon, 11 Feb 2019 19:15:32 GMT
date: Mon, 11 Feb 2019 19:15:32 GMT
cache-control: private, max-age=86400
content-encoding: br
server: ESF
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
alt-svc: quic=":443"; ma=2592000; v="44,43,39"
X-Firefox-Spdy: h2

CORS – Preflighted Request

Above we have seen about the CORS simple request. Unlike simple request the preflight request sends one extra request prior sending the actual request. As discussed above preflight request is sent for the methods which can cause the side effect on server side data, also if any of the conditions that mentioned for simple request doesn’t satisfy.

Methods for which browser makes preflighte request are,

Here is the flow of preflighted request,

CORS Preflight Request Flow
CORS Preflight Request Flow

In the above picture browser from the origin “https://getkt.com” is sending HTTP Post Request for updating profile to host api.getkt.com (imaginary host). Since it is a post request browser makes preflight HTTP POST request with headers.

Origin: https://getkt.com
Access-Control-Request-Method: POST
Access-Control-Request-Headers: X-Csrf-Token, Content-Type

Above headers are sent by browser in OPTIONS request to ask server at different origin that, Does the origin “https://getkt.com” (Origin: https://getkt.com) is having permissions to make a request with method POST (Access-Control-Request-Method: POST) also using headers “X-CSrf-Token, Content-Type” (Access-Control-Request-Headers: X-Csrf-Token, Content-Type).

Server replied to the OPTIONS request(preflight request) with headers,

Access-Control-Allow-Origin: https://getkt.com
Access-Control-Allow-Methods: POST, GET, OPTIONS
Access-Control-Allow-Headers: X-Csrf-Token, Content-Type
Access-Control-Max-Age: 86400

The above response for the OPTIONS request from the server states that, the requested origin “https://getkt.com” is having permission to send request with methods “POST, GET and OPTIONS” including headers “X-Csrf-Token, Content-Type”. With max time to cache this cache this preflight request is 86400.

If it wasn’t the above response from the server. Browser won’t send the actual request and trows an error in the script.

This whole preflight (HTTP OPTIONS) request is the negotiation between browser and server to determine if actual request is having permissions for the resources on remote server along using given headers on actual request.

Here is the simple Javascript snippet to make a request using fetch

 var corsURL = 'https://api.exmaple.com';
var body = {msg: "CORS test"}
var p = fetch(corsURL,
{
method: "POST",
mode: "cors",
headers: {
"Content-Type": "applciation/json",
"X-Csrf-Token": '32kda832da'
},
redirect: 'follow',
body: JSON.stringify(body)
})
p.then(response => response.json())

CORS Headers

CORS Request Haders

Origin

Origin Headers indicates that the origin from which the quest originates
Origin Header is sent with CORS requests includig POST request

Access-Control-Request-Method

The request header Access-Control-Request-Method used in HTTP OPTIONS request. Browser sends this header to server during preflight request, to let server know what kind of method will be used when making actual reqeust after prelight.

Access-Control-Request-Headers

The request header Access-Control-Request-Headers is used in HTTP OPTIONS request. Browser sends this header to server during preflight request, to let server know what kind of headers are used when making actual request after preflight.

CORS Response Headers

Access-Control-Allow-Origin

The Response header Access-Control-Allow-Origin is sent by Cross Origin server in response to preflight request. Which includes either the origin which is sent via Origin header if it is allowed or * if the resource is open to for all origins to access. IF * wildcard is used we can’t pass the credentials to server in request since it open for all it make sense.

Access-Contro-Allow-Origin: <origin> | *

Access-Control-Allow-Methods

The Response header Access-Control-Allow-Method is sent by Cross Origin server in response to preflight request. Which includes Methods that are allowed to access resource in the actual request.

Access-Contro-Allow-Methods: GET, POST, PUT, OPTIONS

Access-Control-Allow-Headers

The Response header Access-Control-Allow-Header is sent by Cross Origin server in response to preflight request. Which includes HTTP Headers that can be used in the actual request.

Access-Control-Allow-Headers: X-Csrf-Token, Content-Type

Access-Control-Expose-Headers

The Response header Access-Control-expose-headers is sent by Cross Origin server in response to preflight request. Which includes headers that browser/script is allowed to access.

Access-Control-Expose-Headers: X-Custom-Header, X-Custom-Cookie

Access-Control-Allow-Credentials

The Response header Access-Control-Allow-Credentials is sent by Cross Origin server in response to preflight request. Which includes whether if actual request can be made with credentials or not. Note that Credential is not supported if the CORS header Access-Control-Allow-Origin is “*“.

Access-Control-Allow-Credentials: true

Access-Control-Max-Age

The Response header Access-Control-Allow-Credentials is sent by Cross Origin server in response to preflight request. Which includes seconds. Which states that, how long the preflight response can be cached

Access-Control-Max-Age: 8400

How to view converted SQL queries of Django ORM

View Django ORM's raw SQL queries

Django provides the ORM which is database and vendor agnostic, as a result we can perform operations on database regardless kind of database running behind.Thus, developer may not need expertise on specifiec database, all he needs is how to play with ORM But, sometimes we may want to see the how Django is converting ORM into raw SQL queries of specific database either for debugging or learning.

There are multiples ways to see converted Queries

View Queries using django.db.connection.queries

You can find queries so far that converted by ORM at django.db.connection.queries

print converted django queries

>>> from django.db  import connection
>>> print(connection.queries)

Access raw SQL query on QuerySet object

Every queryset object has query attribute, which holds the actual SQL query which will be sent to Database. query attribute return query when either print or convert it to str

>>> qs = MyModel.objects.all()
>>> print(qs.query)

Configure Django Logger to log Queries

Django logs debug messages including raw SQL queries it is executing to console. To see these message you have to configure django logger and set DEBUG True

Add following setting to settings.py file of project

DEBUG=True

** DEBUG=True is not recommended in production


Configure django logger and set it’s level DEBUG in settings.py

'django': {
'handlers': ['console'],
'propagate': False,
'level': 'DEBUG'
},

Simple Complete logging setting to log Django logs including queries , Copy and paste the following setting into your setting.py module , so django displays logs messages including queries being sent to DB that are converted from ORM to raw SQL on console

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': '%(asctime)s-%(module)s-%(levelname)s :: %(message)s'
},
'simple': {
'format': '%(levelname)s :: %(message)s'
}
},
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'verbose'
},
},
'loggers': {
'django': {
'handlers': ['console'],
'propagate': False,
'level': 'DEBUG'
},
}
}

Refer more on logging configuration at Django Logging

Database general query logs

All most all DBMS software’s support the logging of queries which are being execute for both debugging and tuning. You can enable the logging for a while to monitor queries from multiple clients.